This introduces a faster implementation of signal and wait operations on NT
events, semaphores, and mutexes, which improves performance to native levels for
a wide variety of games and other applications.
The goal here is similar to the long-standing out-of-tree "esync" and "fsync"
patch sets, but without the flaws that make those patch sets not upstreamable.
The Linux "ntsync" driver is not currently released. It has been accepted into
the trunk Linux tree for 6.14, so barring any extraordinary circumstances, the
API is frozen and it will be released in its current form in about 2 months.
Since it has passed all relevant reviewers on the kernel side, and the API is
all but released, it seems there is no reason any more not to submit the Wine
side to match.
Some important notes:
* This patch set does *not* include any way to disable ntsync support, since
that kind of configuration often seems to be dispreferred where not necessary.
In essence, ntsync should just work everywhere.
Probably the easiest way to effectively disable ntsync, for the purposes of
testing, is to chmod the /dev/ntsync device to prevent its being opened.
Regardless, a Wine switch to disable ntsync can be added simply enough. Note
that it should probably not take the form of a registry key, however, since it
needs to be easily accessible from the server itself.
* It is, generally speaking, not possible for only some objects, or some
processes, to have backing ntsync objects, while others use the old server
path. The esync/fsync patch sets explicitly protected against this by making
sure every process had a consistent view of whether esync was enabled. This is
not provided here, since no switch is provided to toggle ntsync, and it should
not be possible to get into such an inconsistent state without gross
misconfiguration.
* Similarly, no diagnostic messages are provided to note that ntsync is in use,
or not in use. These messages are part of esync/fsync, as well as part of
ntsync "testing" trees unofficially distributed. However, if ntsync is working
correctly, no message should be necessary.
The basic structure is:
* Each type of server object which can be waited on by the client (including
events, semaphores, mutexes, but also other types such as processes, files)
must store an "inproc_sync" object.
This "inproc_sync" object is a full server object (note that this differs from
esync/fsync). A vector and server request is introduced to retrieve an NT
handle to this object from an arbitrary NT handle.
Since the actual ntsync objects are simply distinct file descriptions, we then
call get_handle_fd from the client to retrieve an fd to the object, and then
perform ioctls on it.
* Objects signaled by the server (processes, files, etc) perform ntsync ioctls
on that object. The backing object in all such cases is simply an event.
* Signal and wait operations on the client side attempt to defer to an
"inproc_*" function, falling back to the server implementation if it returns
STATUS_NOT_IMPLEMENTED. This mirrors how in-process synchronization objects
(critical sections, SRW locks, etc) used to be implemented—attempting to use
an architecture-specific "fast_*" function and falling back if it returned
STATUS_NOT_IMPLEMENTED.
* The inproc_sync handles, once retrieved, are cached per-process. This caching
takes a similar form to the fd cache. It does not reuse the same
infrastructure, however.
The primary reason for this is that the fd cache is designed to fit within a
64-bit value and uses 64-bit atomic operations to ensure consistency. However,
we need to store more than 64 bits of information. [We also need to modify
them after caching, in order to correctly implement handle closing—see below.]
The secondary reason is that retrieving the ntsync fd from the inproc_sync
handle itself uses the fd cache.
* In order to keep the Linux driver simple, it does not implement access flags
(EVENT_MODIFY_STATE etc.) Instead, the flags are cached locally and validated
there. This too mirrors the fd cache. Note that this means that a malicious
process can now modify objects it should not be able modify—which is less true
than it is with wineserver—but this is no different from the way other objects
(notably fds) are handled, and would require manual syscalls.
* In order to achieve correct behaviour related to closing objects while they
are used, this patch set essentially relies on refcounting. This is broadly
true of the server as well, but because we need to avoid server calls when
performing object operations, significantly more care must be taken.
In particular, because waits need to be interruptable by signals and then be
restarted, we need the backing ntsync object to remain valid until all users
are done with it. On a process level, this is achieved by letting multiple
processes own handles to the underlying inproc_sync server object.
On a thread level, multiple simultaneous calls need to refcount the process's
local handle. This refcount is stored in the sync object cache. When it
reaches zero, the cache is cleared.
Punting this behaviour to the Linux driver would have introduced a great deal
more complexity, which is best kept in userspace and out of the kernel.
* The cache is, as such, treated as a cache. The penultimate commit, which
introduces client support but does not yet cache the objects, effectively
illustrates this by never actually caching anything, and retrieving a new NT
handle and fd every time.
* Certain waits, on internal handles (such as async, startup_info, completion),
are delegated to the server even when ntsync is used. Those server objects do
not create an underlying ntsync object.
--
This merge request has too many patches to be relayed via email.
Please visit the URL below to see the contents of the merge request.
https://gitlab.winehq.org/wine/wine/-/merge_requests/7226
On Tue Feb 4 22:24:37 2025 +0000, Marc-Aurel Zent wrote:
> Indeed, but this commit essentially inlines the call to
> `__commpage_gettimeofday()` inside `gettimeofday()`, wheres the latter
> still incurs of overhead of 23 CPU cycles before calling into
> `__commpage_gettimeofday()`, if I counted correctly.
> It actually does more than inlining though, `gettimeofday()`, will do a
> syscall if `__commpage_gettimeofday()` fails, and use that value instead.
> This can happen rarely (about 1 in a million in my testing), probably
> when page time values that are updated asynchronously from the kernel
> are read simultaneously.
> As the commit is right now, it effectively tries to read from the
> commpage *twice*, making that chance of doing a syscall astronomically low.
> Also it is still measurably faster to use `__commpage_gettimeofday()`
> directly (benchmarked at 10000000 iterations through Rosetta):
> ```
> clock_gettime() Duration: 0.290506 seconds
> gettimeofday() Duration: 0.233024 seconds
> __commpage_gettimeofday() Duration: 0.217468 seconds
> ```
IMO, this feels like a micro-optimization that isn't worth the complexity and risk to save 23 cycles per call. If it makes a meaningful difference in a game benchmark it could be worth it, but otherwise it feels better to chase bigger gains.
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/7256#note_93758
"DIR /o" (no order specifier after /o) hasn't functioned the same as it does on Windows. This change makes the functionality of /o by itself equivalent to behavior on Windows.
--
v23: programs/cmd: Cleanup DIR /O logic and remove @todo_wine@ from working tests.
programs/cmd/tests: Add tests for command DIR /O.
https://gitlab.winehq.org/wine/wine/-/merge_requests/7131
The software Helicon Filter 5 crashes at startup when it tries to query
the registry entry `SystemBiosDate` under `HKEY_LOCAL_MACHINE\Hardware\Description\System`, which doesn't exist in the wine registry.
That part of the registry is volatile, so the workaround of just adding that key is short lived.
In this commit I just added a dummy date (01/01/70) for simplicity, not sure if that's ok?
There's code in wineboot.c which sets the actual BIOS date into the `BIOSReleaseDate` key. That could potentially be used, but it has a different format: 01/01/1970 (four digits for the year instead of two)
I've found no official documentation of the `SystemBiosDate` key. The only place I've found is in an example on this page: https://learn.microsoft.com/en-us/windows-hardware/drivers/debuggercmds/-reg
<details><summary>Wine log from startup crash when querying SystemBiosDate</summary>
<pre>
0024:Call advapi32.RegOpenKeyExA(80000002,00810cdc "HARDWARE\\DESCRIPTION\\System",00000000,00020019,00e6fb68) ret=00440607
0024:Call ntdll.RtlRunOnceExecuteOnce(7b69ec60,7b65bad0,00000000,00000000) ret=7b6106d9
0024:Ret ntdll.RtlRunOnceExecuteOnce() retval=00000000 ret=7b6106d9
0024:Call ntdll.wcscpy(00e6f9ac,7b69ec94 L"") ret=7b60ce6b
0024:Ret ntdll.wcscpy() retval=00e6f9ac ret=7b60ce6b
0024:Call ntdll.RtlInitAnsiString(00e6fadc,00810cdc "HARDWARE\\DESCRIPTION\\System") ret=7b62f0bc
0024:Ret ntdll.RtlInitAnsiString() retval=0000001c ret=7b62f0bc
0024:Call ntdll.RtlAnsiStringToUnicodeString(7ffc2bf8,00e6fadc,00000000) ret=7b62f0e1
0024:Ret ntdll.RtlAnsiStringToUnicodeString() retval=00000000 ret=7b62f0e1
0024:Call ntdll.NtOpenKeyEx(00e6fb68,00020019,00e6f9a8,00000000) ret=7b65e875
0024:Ret ntdll.NtOpenKeyEx() retval=00000000 ret=7b65e875
0024:Call ntdll.RtlNtStatusToDosError(00000000) ret=7b62f0f1
0024:Ret ntdll.RtlNtStatusToDosError() retval=00000000 ret=7b62f0f1
0024:Ret advapi32.RegOpenKeyExA() retval=00000000 ret=00440607
0024:Call advapi32.RegQueryValueExA(000000f8,00810cc4 "SystemBiosDate",00000000,00e6fb58,00000000,00e6fb74) ret=00440c43
0024:Call ntdll.RtlInitAnsiString(00e6f9b0,00810cc4 "SystemBiosDate") ret=7b6305f6
0024:Ret ntdll.RtlInitAnsiString() retval=0000000f ret=7b6305f6
0024:Call ntdll.RtlAnsiStringToUnicodeString(00e6f9b8,00e6f9b0,00000001) ret=7b630614
0024:Ret ntdll.RtlAnsiStringToUnicodeString() retval=00000000 ret=7b630614
0024:Call ntdll.NtQueryValueKey(000000f8,00e6f9b8,00000002,00e6f9c0,00000100,00e6f9a8) ret=7b630695
0024:Ret ntdll.NtQueryValueKey() retval=c0000034 ret=7b630695
0024:Call ntdll.RtlFreeUnicodeString(00e6f9b8) ret=7b6306c0
0024:Ret ntdll.RtlFreeUnicodeString() retval=00000001 ret=7b6306c0
0024:Call ntdll.RtlNtStatusToDosError(c0000034) ret=7b6306d2
0024:Ret ntdll.RtlNtStatusToDosError() retval=00000002 ret=7b6306d2
0024:Ret advapi32.RegQueryValueExA() retval=00000002 ret=00440c43
0024:Call advapi32.RegCloseKey(000000f8) ret=00440433
0024:Call ntdll.NtClose(000000f8) ret=7b628979
0024:Ret ntdll.NtClose() retval=00000000 ret=7b628979
0024:Call ntdll.RtlNtStatusToDosError(00000000) ret=7bcebd50
0024:Ret ntdll.RtlNtStatusToDosError() retval=00000000 ret=7bcebd50
0024:Ret advapi32.RegCloseKey() retval=00000000 ret=00440433
0024:Call user32.LoadStringA(00400000,0000fff0,00e6eb54,00001000) ret=0040879f
0024:Call ntdll.LdrFindResource_U(00400000,00e6ea4c,00000003,00e6ea38) ret=7b5fbb3b
0024:Ret ntdll.LdrFindResource_U() retval=00000000 ret=7b5fbb3b
0024:Call ntdll.LdrAccessResource(00400000,00b033e0,00e6eac0,00000000) ret=7b6148f6
0024:Ret ntdll.LdrAccessResource() retval=00000000 ret=7b6148f6
0024:Ret user32.LoadStringA() retval=00000021 ret=0040879f
0024:Call KERNEL32.RaiseException(0eedfade,00000001,00000007,00e6fb88) ret=00417362
0024:trace:seh:dispatch_exception code=eedfade (unknown) flags=1 addr=7B627448
0024:trace:seh:dispatch_exception info[0]=00417362
0024:trace:seh:dispatch_exception info[1]=02B80430
0024:trace:seh:dispatch_exception info[2]=0041625C
0024:trace:seh:dispatch_exception info[3]=00E6FBB8
0024:trace:seh:dispatch_exception info[4]=00000000
0024:trace:seh:dispatch_exception info[5]=00E6FDF8
0024:trace:seh:dispatch_exception info[6]=00E6FBA4
0024:trace:seh:dispatch_exception eip=7b627448 esp=00e6fae0 ebp=00e6fb38 eflags=00000246
0024:trace:seh:dispatch_exception eax=00e6fae0 ebx=00000018 ecx=00e6faf4 edx=00e6fb88
0024:trace:seh:dispatch_exception esi=00e6fdf8 edi=00e6fb50 cs=0023 ds=002b es=002b fs=0063 gs=006b ss=002b
0024:trace:seh:call_seh_handlers calling handler at 00810CAC code=eedfade flags=1
0024:trace:seh:call_seh_handlers handler at 00810CAC returned 1
0024:trace:seh:call_seh_handlers calling handler at 00874F3A code=eedfade flags=1
0024:trace:seh:call_seh_handlers handler at 00874F3A returned 1
0024:trace:seh:call_seh_handlers calling handler at 0096B5C7 code=eedfade flags=1
0024:trace:seh:call_seh_handlers handler at 0096B5C7 returned 1
0024:trace:seh:call_seh_handlers calling handler at 004059FC code=eedfade flags=1
0024:Call KERNEL32.UnhandledExceptionFilter(00e6ef18) ret=00405a20
0024:Call ntdll.NtQueryInformationProcess(ffffffff,0000000c,00e6ee84,00000004,00000000) ret=7b602304
0024:Ret ntdll.NtQueryInformationProcess() retval=00000000 ret=7b602304
0024:Call ntdll.NtCreateEvent(00e6eeac,001f0003,00e6eeb0,00000000,00000000) ret=7b642f85
0024:Ret ntdll.NtCreateEvent() retval=00000000 ret=7b642f85
wine: Unhandled exception 0x0eedfade in thread 24 at address 7B627448 (thread 0024), starting debugger...
</pre>
</details>
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/7271
This MR enforces alignment of TLS slots as described in PE file.
--
v3: ntdll: Let layout of TLS vector be closer to native.
ntdll: Enforce the alignment of TLS directory entries.
kernel32: Add test for TLS memory layout.
kernel32/tests: Add a test about TLS slot alignment.
https://gitlab.winehq.org/wine/wine/-/merge_requests/7251
Otherwise sending ^C to a running ARM64 process crashes it randomly.
Note that 0x110 is prev_frame, the field that immediately precedes
syscall_cfa.
Fixes: ca79629b3e436c8799281319dd313249bf05b9b5
--
v2: ntdll: Fix syscall_cfa offset in user_mode_abort_thread for ARM64.
https://gitlab.winehq.org/wine/wine/-/merge_requests/7268
Otherwise sending ^C to a running ARM64 process crashes it randomly.
Note that 0x110 is prev_frame, the field that immediately precedes
syscall_cfa.
Fixes: ca79629b3e436c8799281319dd313249bf05b9b5
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/7268
posix_fallocate() looks like a better fit when increasing allocation size but you'd need to check the file size first, which introduces a race condition. posix_fallocate() will also increase the file size if the file is truncated by another thread.
--
https://gitlab.winehq.org/wine/wine/-/merge_requests/7115#note_93690