February 2025 - Wine-GitLab

[PATCH v3 0/26] MR7226: ntdll: Implement in-process synchronization via the Linux "ntsync" driver.
by Elizabeth Figura (＠zfigura) Feb. 16, 2025

Feb. 16, 2025

This introduces a faster implementation of signal and wait operations on NT events, semaphores, and mutexes, which improves performance to native levels for a wide variety of games and other applications. The goal here is similar to the long-standing out-of-tree "esync" and "fsync" patch sets, but without the flaws that make those patch sets not upstreamable. The Linux "ntsync" driver is not currently released. It has been accepted into the trunk Linux tree for 6.14, so barring any extraordinary circumstances, the API is frozen and it will be released in its current form in about 2 months. Since it has passed all relevant reviewers on the kernel side, and the API is all but released, it seems there is no reason any more not to submit the Wine side to match. Some important notes: * This patch set does *not* include any way to disable ntsync support, since that kind of configuration often seems to be dispreferred where not necessary. In essence, ntsync should just work everywhere. Probably the easiest way to effectively disable ntsync, for the purposes of testing, is to chmod the /dev/ntsync device to prevent its being opened. Regardless, a Wine switch to disable ntsync can be added simply enough. Note that it should probably not take the form of a registry key, however, since it needs to be easily accessible from the server itself. * It is, generally speaking, not possible for only some objects, or some processes, to have backing ntsync objects, while others use the old server path. The esync/fsync patch sets explicitly protected against this by making sure every process had a consistent view of whether esync was enabled. This is not provided here, since no switch is provided to toggle ntsync, and it should not be possible to get into such an inconsistent state without gross misconfiguration. * Similarly, no diagnostic messages are provided to note that ntsync is in use, or not in use. These messages are part of esync/fsync, as well as part of ntsync "testing" trees unofficially distributed. However, if ntsync is working correctly, no message should be necessary. The basic structure is: * Each type of server object which can be waited on by the client (including events, semaphores, mutexes, but also other types such as processes, files) must store an "inproc_sync" object. This "inproc_sync" object is a full server object (note that this differs from esync/fsync). A vector and server request is introduced to retrieve an NT handle to this object from an arbitrary NT handle. Since the actual ntsync objects are simply distinct file descriptions, we then call get_handle_fd from the client to retrieve an fd to the object, and then perform ioctls on it. * Objects signaled by the server (processes, files, etc) perform ntsync ioctls on that object. The backing object in all such cases is simply an event. * Signal and wait operations on the client side attempt to defer to an "inproc_\*" function, falling back to the server implementation if it returns STATUS_NOT_IMPLEMENTED. This mirrors how in-process synchronization objects (critical sections, SRW locks, etc) used to be implemented—attempting to use an architecture-specific "fast_\*" function and falling back if it returned STATUS_NOT_IMPLEMENTED. * The inproc_sync handles, once retrieved, are cached per-process. This caching takes a similar form to the fd cache. It does not reuse the same infrastructure, however. The primary reason for this is that the fd cache is designed to fit within a 64-bit value and uses 64-bit atomic operations to ensure consistency. However, we need to store more than 64 bits of information. [We also need to modify them after caching, in order to correctly implement handle closing—see below.] The secondary reason is that retrieving the ntsync fd from the inproc_sync handle itself uses the fd cache. * In order to keep the Linux driver simple, it does not implement access flags (EVENT_MODIFY_STATE etc.) Instead, the flags are cached locally and validated there. This too mirrors the fd cache. Note that this means that a malicious process can now modify objects it should not be able modify—which is less true than it is with wineserver—but this is no different from the way other objects (notably fds) are handled, and would require manual syscalls. * In order to achieve correct behaviour related to closing objects while they are used, this patch set essentially relies on refcounting. This is broadly true of the server as well, but because we need to avoid server calls when performing object operations, significantly more care must be taken. In particular, because waits need to be interruptable by signals and then be restarted, we need the backing ntsync object to remain valid until all users are done with it. On a process level, this is achieved by letting multiple processes own handles to the underlying inproc_sync server object. On a thread level, multiple simultaneous calls need to refcount the process's local handle. This refcount is stored in the sync object cache. When it reaches zero, the cache is cleared. Punting this behaviour to the Linux driver would have introduced a great deal more complexity, which is best kept in userspace and out of the kernel. * The cache is, as such, treated as a cache. The penultimate commit, which introduces client support but does not yet cache the objects, effectively illustrates this by never actually caching anything, and retrieving a new NT handle and fd every time. * Certain waits, on internal handles (such as async, startup_info, completion), are delegated to the server even when ntsync is used. Those server objects do not create an underlying ntsync object. -- v3: ntdll: Cache in-process synchronization objects. ntdll: Use server_wait_for_object() when waiting on only the queue object. ntdll: Use in-process synchronization objects. ntdll: Introduce a helper to wait on an internal server handle. server: Allow creating an event object for client-side user APC signaling. server: Introduce select_inproc_queue and unselect_inproc_queue requests. server: Add a request to retrieve the in-process synchronization object from a handle. server: Create in-process synchronization objects for fd-based objects. server: Create in-process synchronization objects for timers. server: Create in-process synchronization objects for threads. server: Create in-process synchronization objects for message queues. server: Create in-process synchronization objects for jobs. server: Create in-process synchronization objects for processes. server: Create in-process synchronization objects for keyed events. server: Create in-process synchronization objects for device managers. server: Create in-process synchronization objects for debug objects. server: Create in-process synchronization objects for console servers. server: Create in-process synchronization objects for consoles. server: Create in-process synchronization objects for completion ports. server: Create in-process synchronization objects for mutexes. server: Create in-process synchronization objects for semaphores. server: Create in-process synchronization objects for events. server: Add an object operation to retrieve an in-process synchronization object. ntdll: Retrieve and cache an ntsync device in wait calls. ntdll: Add stub functions for in-process synchronization. ntdll: Add some traces to synchronization methods. This merge request has too many patches to be relayed via email. Please visit the URL below to see the contents of the merge request. https://gitlab.winehq.org/wine/wine/-/merge_requests/7226

10 26

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by Joe Souza (＠JoeS209) Feb. 16, 2025

Feb. 16, 2025

On Sun Feb 16 16:37:14 2025 +0000, eric pouech wrote: > this shouldn't set errorlevel but return_code > and likely using ERROR_FILE_NOT_FOUND to mark that listing was empty My head is starting to spin here. That is existing code that I haven't touched. I'll fix along with the other issues though... -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94829

1 0

Re: [PATCH v2 0/1] MR7339: ntdll: implement create_logical_proc_info on FreeBSD.
by Brendan Shanks (＠bshanks) Feb. 16, 2025

Feb. 16, 2025

Another possibility is to use `libbsdxml`, a renamed expat that FreeBSD includes (although the man page says that “The libbsdxml library is intended to be used within the FreeBSD base system only. Use of the libbsdxml library for other purposes is not supported and discouraged.” But expat’s API is quite stable, and avahi also has the option to use bsdxml.) Otherwise, maybe just depend on a system-installed expat or libxml2 on FreeBSD? -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7339#note_94826

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

globally it looks better a couple of stylistic remarks, but you need to go further for not using errorlevel inside the function and its helper perhaps, it's worthwhile to split the patch in two: * first one to no longer use error level internally, adapt list_directory prototype... (in the details, it's fine to use return errorlevel = ERROR\_\* while parsing the arguments in case of error, but when this is done all the rest of the code should use only the return_code (and set errorlevel when exiting WCMD_dir)) * the second one to plug in the ctrl-c handling and breaking from the various loops -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94825

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

eric pouech (@epo) commented about programs/cmd/directory.c: > * FIXME: Assumes 24-line display for the /P qualifier. > */ > > -static DIRECTORY_STACK *WCMD_list_directory (DIRECTORY_STACK *inputparms, int level) { > +static DIRECTORY_STACK *WCMD_list_directory (DIRECTORY_STACK *inputparms, int level, RETURN_CODE *return_code) { nitpick: IMO it's more readable to return a RETURN_CODE, and pass a DIRECTORY_STACK\*\* to update the pointer (you likely did it this way to minimize change, but let's take the opportunity of a change to sanitize a bit the current spaghetti code) -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94822

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

eric pouech (@epo) commented about programs/cmd/directory.c: > thisDir = tempDir; > } > } > + /* Free remaining structures if we broke out of previous loop due to error. */ > + while (dirStack != NULL) { this likely could be folded in code line 510, by inserting between 510 & 511: if (return_code != NO_ERROR) dirStack = NULL; -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94821

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

eric pouech (@epo) commented about programs/cmd/directory.c: > } > > /* Trailer Information */ > - if (trailerReqd) { > + if (trailerReqd && return_code == NO_ERROR) { > WCMD_dir_trailer(prevEntry->dirName); > } > > - if (num_empty && !num_with_data) > + if (return_code == STATUS_CONTROL_C_EXIT) actually, we could (later on) have other values as return code... so IMO this should rather be written as: `if (return_code != NO_ERROR || (num_empty && !num_with_data))` ` errorlevel = ERROR_INVALID_FUNCTION;` `else` ` errorlevel = NO_ERROR;` -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94820

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

eric pouech (@epo) commented about programs/cmd/directory.c: > /* Clear any errors from previous invocations, and process it */ > errorlevel = NO_ERROR; > prevEntry = thisEntry; > - thisEntry = WCMD_list_directory (thisEntry, 0); > + thisEntry = WCMD_list_directory (thisEntry, 0, &return_code); > if (errorlevel) this should use return_code not error level -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94819

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

eric pouech (@epo) commented about programs/cmd/directory.c: > if ((file_total + dir_total == 0) && (level == 0)) { > SetLastError (ERROR_FILE_NOT_FOUND); > WCMD_print_error (); > errorlevel = ERROR_INVALID_FUNCTION; this shouldn't set errorlevel but return_code and likely using ERROR_FILE_NOT_FOUND to mark that listing was empty -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94818

1 0

Re: [PATCH v10 0/1] MR7322: programs/cmd: Implement ability to abort lengthy directory operations via Ctrl-C.
by eric pouech (＠epo) Feb. 16, 2025

Feb. 16, 2025

eric pouech (@epo) commented about programs/cmd/directory.c: > WCHAR fname[MAX_PATH]; > WCHAR ext[MAX_PATH]; > unsigned num_empty = 0, num_with_data = 0; > + RETURN_CODE return_code = NO_ERROR; > > errorlevel = NO_ERROR; this should not be necessary (in fact starting from here up the exit of the function, you should only deal with return_code... errorlevel should be set upon exiting) -- https://gitlab.winehq.org/wine/wine/-/merge_requests/7322#note_94817

1 0