March 2024 - Wine-Devel - list.winehq.org

[PATCH v3 00/30] NT synchronization primitive driver

by Elizabeth Figura

This patch series introduces a new char misc driver, /dev/ntsync, which is used to implement Windows NT synchronization primitives. == Background == The Wine project emulates the Windows API in user space. One particular part of that API, namely the NT synchronization primitives, have historically been implemented via RPC to a dedicated "kernel" process. However, more recent applications use these APIs more strenuously, and the overhead of RPC has become a bottleneck. The NT synchronization APIs are too complex to implement on top of existing primitives without sacrificing correctness. Certain operations, such as NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require direct control over the underlying wait queue, and implementing a wait queue sufficiently robust for Wine in user space is not possible. This proposed driver, therefore, implements the problematic interfaces directly in the Linux kernel. This driver was presented at Linux Plumbers Conference 2023. For those further interested in the history of synchronization in Wine and past attempts to solve this problem in user space, a recording of the presentation can be viewed here: https://www.youtube.com/watch?v=NjU4nyWyhU8 == Performance == The gain in performance varies wildly depending on the application in question and the user's hardware. For some games NT synchronization is not a bottleneck and no change can be observed, but for others frame rate improvements of 50 to 150 percent are not atypical. The following table lists frame rate measurements from a variety of games on a variety of hardware, taken by users Dmitry Skvortsov, FuzzyQuils, OnMars, and myself: Game Upstream ntsync improvement =========================================================================== Anger Foot 69 99 43% Call of Juarez 99.8 224.1 125% Dirt 3 110.6 860.7 678% Forza Horizon 5 108 160 48% Lara Croft: Temple of Osiris 141 326 131% Metro 2033 164.4 199.2 21% Resident Evil 2 26 77 196% The Crew 26 51 96% Tiny Tina's Wonderlands 130 360 177% Total War Saga: Troy 109 146 34% =========================================================================== == Patches == The intended semantics of the patches are broadly intended to match those of the corresponding Windows functions. For those not already familiar with the Windows functions (or their undocumented behaviour), patch 31/31 provides a detailed specification, and individual patches also include a brief description of the API they are implementing. The patches making use of this driver in Wine can be retrieved or browsed here: https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync5 == Implementation == Some aspects of the implementation may deserve particular comment: * In the interest of performance, each object is governed only by a single spinlock. However, NTSYNC_IOC_WAIT_ALL requires that the state of multiple objects be changed as a single atomic operation. In order to achieve this, we first take a device-wide lock ("wait_all_lock") any time we are going to lock more than one object at a time. The maximum number of objects that can be used in a vectored wait, and therefore the maximum that can be locked simultaneously, is 64. This number is NT's own limit. The acquisition of multiple spinlocks will degrade performance. This is a conscious choice, however. Wait-for-all is known to be a very rare operation in practice, especially with counts that approach the maximum, and it is the intent of the ntsync driver to optimize wait-for-any at the expense of wait-for-all as much as possible. * NT mutexes are tied to their threads on an OS level, and the kernel includes builtin support for "robust" mutexes. In order to keep the ntsync driver self-contained and avoid touching more code than necessary, it does not hook into task exit nor use pids. Instead, the user space emulator is expected to manage thread IDs and pass them as an argument to any relevant functions; this is the "owner" field of ntsync_wait_args and ntsync_mutex_args. When the emulator detects that a thread dies, it should therefore call NTSYNC_IOC_MUTEX_KILL on any open mutexes. * ntsync is module-capable mostly because there was nothing preventing it, and because it aided development. It is not a hard requirement, though. == Previous versions == Changes from v2: * Check the result of fget() for NULL. * Squash patch 31 (introducing the NTSYNC_WAIT_REALTIME flag) into patch 4, per Arnd Bergmann. * Use atomic_try_cmpxchg() instead of atomic_cmpxchg(), per off-list review from Uros Bizjak. * Link to v2: https://lore.kernel.org/lkml/20240219223833.95710-1-zfigura(a)codeweavers.c… * Link to v1: https://lore.kernel.org/lkml/20240214233645.9273-1-zfigura(a)codeweavers.co… * Link to RFC v2: https://lore.kernel.org/lkml/20240131021356.10322-1-zfigura(a)codeweavers.c… * Link to RFC v1: https://lore.kernel.org/lkml/20240124004028.16826-1-zfigura(a)codeweavers.c… Elizabeth Figura (30): ntsync: Introduce the ntsync driver and character device. ntsync: Introduce NTSYNC_IOC_CREATE_SEM. ntsync: Introduce NTSYNC_IOC_SEM_POST. ntsync: Introduce NTSYNC_IOC_WAIT_ANY. ntsync: Introduce NTSYNC_IOC_WAIT_ALL. ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX. ntsync: Introduce NTSYNC_IOC_MUTEX_UNLOCK. ntsync: Introduce NTSYNC_IOC_MUTEX_KILL. ntsync: Introduce NTSYNC_IOC_CREATE_EVENT. ntsync: Introduce NTSYNC_IOC_EVENT_SET. ntsync: Introduce NTSYNC_IOC_EVENT_RESET. ntsync: Introduce NTSYNC_IOC_EVENT_PULSE. ntsync: Introduce NTSYNC_IOC_SEM_READ. ntsync: Introduce NTSYNC_IOC_MUTEX_READ. ntsync: Introduce NTSYNC_IOC_EVENT_READ. ntsync: Introduce alertable waits. selftests: ntsync: Add some tests for semaphore state. selftests: ntsync: Add some tests for mutex state. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for manual-reset event state. selftests: ntsync: Add some tests for auto-reset event state. selftests: ntsync: Add some tests for wakeup signaling with events. selftests: ntsync: Add tests for alertable waits. selftests: ntsync: Add some tests for wakeup signaling via alerts. selftests: ntsync: Add a stress test for contended waits. maintainers: Add an entry for ntsync. docs: ntsync: Add documentation for the ntsync uAPI. Documentation/userspace-api/index.rst | 1 + .../userspace-api/ioctl/ioctl-number.rst | 2 + Documentation/userspace-api/ntsync.rst | 399 +++++ MAINTAINERS | 9 + drivers/misc/Kconfig | 11 + drivers/misc/Makefile | 1 + drivers/misc/ntsync.c | 1166 ++++++++++++++ include/uapi/linux/ntsync.h | 62 + tools/testing/selftests/Makefile | 1 + .../testing/selftests/drivers/ntsync/Makefile | 8 + tools/testing/selftests/drivers/ntsync/config | 1 + .../testing/selftests/drivers/ntsync/ntsync.c | 1407 +++++++++++++++++ 12 files changed, 3068 insertions(+) create mode 100644 Documentation/userspace-api/ntsync.rst create mode 100644 drivers/misc/ntsync.c create mode 100644 include/uapi/linux/ntsync.h create mode 100644 tools/testing/selftests/drivers/ntsync/Makefile create mode 100644 tools/testing/selftests/drivers/ntsync/config create mode 100644 tools/testing/selftests/drivers/ntsync/ntsync.c base-commit: 4cece764965020c22cff7665b18a012006359095 -- 2.43.0

March 29, 2024

List failing (?!?!)

by David Kahurani

Hello, I am subscribed to receive mail from the list wine(a)gitlab.winehq.org However, I notice this morning that all mail from last night's merger are missing. Did the list fail or something I don't understand? Regards David.

March 27, 2024

Wine staging 9.5 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.5 (435 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * shell32: Append .exe when registry lookup fails first time. * wined3d: Use bindless textures for GLSL shaders. * kernel32/tests: Add tests for job object accounting. * shell32: Fix SHFileOperation(FO_MOVE) for creating subdirectories. * user32: Added LoadKeyboardLayoutEx stub. Removed (No longer required). * ws2_32-TransmitFile * ntdll-Threading Added: * None. Updated: * vkd3d-latest NOTE: widl-SLTG_Typelib_Support has been disabled due and is currently being upstreamed. Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

March 23, 2024

Re: [PATCH v2 04/31] ntsync: Introduce NTSYNC_IOC_WAIT_ANY.

by Elizabeth Figura

On Monday, 19 February 2024 16:38:06 CDT Elizabeth Figura wrote: > +static struct ntsync_obj *get_obj(struct ntsync_device *dev, int fd) > +{ > + struct file *file = fget(fd); > + struct ntsync_obj *obj; > + > + if (file->f_op != &ntsync_obj_fops) { > + fput(file); > + return NULL; > + } I just noticed during self-review that this fails to check the result of fget() for NULL :-/ I'll fix this next revision. > + > + obj = file->private_data; > + if (obj->dev != dev) { > + fput(file); > + return NULL; > + } > + > + return obj; > +}

March 11, 2024

Wine staging 9.4 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download vkd3d has been updated to 1.11 For a full list of changes, please follow the link below. https://gitlab.winehq.org/wine/vkd3d/-/releases/vkd3d-1.11 NOTE: Upstream Wine was updated to this version as well. We have continued with keeping wine-staging to latest vkd3d. Summary since last release * Rebased to current wine 9.4 (462 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * shell32: Implement NewMenu with new folder item. * shell32: Implement insert/paste for item context menus. * shell32: Recognize cut/copy/paste string verbs in item menu - context menu. * shell32: Implement the "runas" verb. * shell32: Add support for setting/getting PREFERREDDROPEFFECT in IDataObject. * shell32: Add parameter to ISFHelper::DeleteItems to allow deleting files without confirmation. * shell32: Remove source files when using cut in the context menu. * wine.inf: Add 'New' context menu handler entry for directories. * msi: Do not sign extend after multiplying. * wine.inf: Set the EnableLUA value to 1. * msi: Create the custom action server as an elevated process. * ntdll: Implement process token elevation through manifests. * ntdll: Always start the initial process through start.exe. * ntdll: Elevate processes if requested in RtlCreateUserProcess(). * user32: Start explorer.exe using limited rights. * programs/runas: Basic implementation for starting processes with a different trustlevel. * server: Create processes using a limited administrator token by default. * kernelbase: Elevate processes if requested in CreateProcessInternal(). Removed (No longer required). * server-File_Permissions * mscoree-CorValidateImage (Fixed uptream already). Added: * None. Updated: * vkd3d-latest * kernel32-CopyFileEx * ntdll-Junction_Points * ntdll-WRITECOPY * user32-rawinput-mouse Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

March 9, 2024

Re: Staging update drop part 2

by Elizabeth Figura

On Thursday, 7 March 2024 14:51:57 CST DodoGTA GT wrote: > Hello, > > I managed to prepare another round of patches for wine-staging (the > last one is actually a Wine patch though and it's a bit hacky too so > I'm not sure if it's a good solution) Yeah, 4/4 is pretty much wrong. 3/4 doesn't look right either, or at least, it's suspicious that it matters. We don't pass application buffers directly to a system call anymore; there's a memcpy through random_buf in advapi32. Technically if someone put a write watch on advapi32's .data section that would make a difference, but that seems unlikely, and it's certainly not the case for services.exe. It needs more investigation. I've applied 1/4 and 2/4, but again, in the future *please* stop sending diffs of diffs; they are much more time-consuming to review and apply. Please just send a single diff, or better yet apply the energy towards upstreaming these patches instead of just rebasing them. Thanks! --Zeb

March 8, 2024

Staging update drop part 2

by DodoGTA GT

Hello, I managed to prepare another round of patches for wine-staging (the last one is actually a Wine patch though and it's a bit hacky too so I'm not sure if it's a good solution) Thanks

March 7, 2024