February 2024 - Wine-Devel - list.winehq.org

[PATCH 01/31] ntsync: Introduce the ntsync driver and character device.

by Elizabeth Figura

ntsync uses a misc device as the simplest and least intrusive uAPI interface. Each file description on the device represents an isolated NT instance, intended to correspond to a single NT virtual machine. Signed-off-by: Elizabeth Figura <zfigura(a)codeweavers.com> --- drivers/misc/Kconfig | 9 ++++++++ drivers/misc/Makefile | 1 + drivers/misc/ntsync.c | 52 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+) create mode 100644 drivers/misc/ntsync.c diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 4fb291f0bf7c..bdd8a71bd853 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -504,6 +504,15 @@ config OPEN_DICE measured boot flow. Userspace can use CDIs for remote attestation and sealing. +config NTSYNC + tristate "NT synchronization primitive emulation" + help + This module provides kernel support for emulation of Windows NT + synchronization primitives. It is not a hardware driver. + + To compile this driver as a module, choose M here: the + module will be called ntsync. + If unsure, say N. config VCPU_STALL_DETECTOR diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index ea6ea5bbbc9c..153a3f4837e8 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -59,6 +59,7 @@ obj-$(CONFIG_PVPANIC) += pvpanic/ obj-$(CONFIG_UACCE) += uacce/ obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o +obj-$(CONFIG_NTSYNC) += ntsync.o obj-$(CONFIG_HI6421V600_IRQ) += hi6421v600-irq.o obj-$(CONFIG_OPEN_DICE) += open-dice.o obj-$(CONFIG_GP_PCI1XXXX) += mchp_pci1xxxx/ diff --git a/drivers/misc/ntsync.c b/drivers/misc/ntsync.c new file mode 100644 index 000000000000..e4969ef90722 --- /dev/null +++ b/drivers/misc/ntsync.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * ntsync.c - Kernel driver for NT synchronization primitives + * + * Copyright (C) 2024 Elizabeth Figura + */ + +#include <linux/fs.h> +#include <linux/miscdevice.h> +#include <linux/module.h> + +#define NTSYNC_NAME "ntsync" + +static int ntsync_char_open(struct inode *inode, struct file *file) +{ + return nonseekable_open(inode, file); +} + +static int ntsync_char_release(struct inode *inode, struct file *file) +{ + return 0; +} + +static long ntsync_char_ioctl(struct file *file, unsigned int cmd, + unsigned long parm) +{ + switch (cmd) { + default: + return -ENOIOCTLCMD; + } +} + +static const struct file_operations ntsync_fops = { + .owner = THIS_MODULE, + .open = ntsync_char_open, + .release = ntsync_char_release, + .unlocked_ioctl = ntsync_char_ioctl, + .compat_ioctl = compat_ptr_ioctl, + .llseek = no_llseek, +}; + +static struct miscdevice ntsync_misc = { + .minor = MISC_DYNAMIC_MINOR, + .name = NTSYNC_NAME, + .fops = &ntsync_fops, +}; + +module_misc_device(ntsync_misc); + +MODULE_AUTHOR("Elizabeth Figura"); +MODULE_DESCRIPTION("Kernel driver for NT synchronization primitives"); +MODULE_LICENSE("GPL"); -- 2.43.0

Feb. 14, 2024

[PATCH 00/31] NT synchronization primitive driver

by Elizabeth Figura

This patch series introduces a new char misc driver, /dev/ntsync, which is used to implement Windows NT synchronization primitives. This was previously submitted as an RFC [1]. Since there were no major changes requested to the last RFC revision, I've stripped the RFC prefix. [1] https://lore.kernel.org/lkml/20240131021356.10322-1-zfigura(a)codeweavers.c… == Background == The Wine project emulates the Windows API in user space. One particular part of that API, namely the NT synchronization primitives, have historically been implemented via RPC to a dedicated "kernel" process. However, more recent applications use these APIs more strenuously, and the overhead of RPC has become a bottleneck. The NT synchronization APIs are too complex to implement on top of existing primitives without sacrificing correctness. Certain operations, such as NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require direct control over the underlying wait queue, and implementing a wait queue sufficiently robust for Wine in user space is not possible. This proposed driver, therefore, implements the problematic interfaces directly in the Linux kernel. This driver was presented at Linux Plumbers Conference 2023. For those further interested in the history of synchronization in Wine and past attempts to solve this problem in user space, a recording of the presentation can be viewed here: https://www.youtube.com/watch?v=NjU4nyWyhU8 == Performance == The gain in performance varies wildly depending on the application in question and the user's hardware. For some games NT synchronization is not a bottleneck and no change can be observed, but for others frame rate improvements of 50 to 150 percent are not atypical. The following table lists frame rate measurements from a variety of games on a variety of hardware, taken by users Dmitry Skvortsov, FuzzyQuils, OnMars, and myself: Game Upstream ntsync improvement =========================================================================== Anger Foot 69 99 43% Call of Juarez 99.8 224.1 125% Dirt 3 110.6 860.7 678% Forza Horizon 5 108 160 48% Lara Croft: Temple of Osiris 141 326 131% Metro 2033 164.4 199.2 21% Resident Evil 2 26 77 196% The Crew 26 51 96% Tiny Tina's Wonderlands 130 360 177% Total War Saga: Troy 109 146 34% =========================================================================== == Patches == The intended semantics of the patches are broadly intended to match those of the corresponding Windows functions. For those not already familiar with the Windows functions (or their undocumented behaviour), patch 31/31 provides a detailed specification, and individual patches also include a brief description of the API they are implementing. The patches making use of this driver in Wine can be retrieved or browsed here: https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync5 == Implementation == Some aspects of the implementation may deserve particular comment: * In the interest of performance, each object is governed only by a single spinlock. However, NTSYNC_IOC_WAIT_ALL requires that the state of multiple objects be changed as a single atomic operation. In order to achieve this, we first take a device-wide lock ("wait_all_lock") any time we are going to lock more than one object at a time. The maximum number of objects that can be used in a vectored wait, and therefore the maximum that can be locked simultaneously, is 64. This number is NT's own limit. The acquisition of multiple spinlocks will degrade performance. This is a conscious choice, however. Wait-for-all is known to be a very rare operation in practice, especially with counts that approach the maximum, and it is the intent of the ntsync driver to optimize wait-for-any at the expense of wait-for-all as much as possible. * NT mutexes are tied to their threads on an OS level, and the kernel includes builtin support for "robust" mutexes. In order to keep the ntsync driver self-contained and avoid touching more code than necessary, it does not hook into task exit nor use pids. Instead, the user space emulator is expected to manage thread IDs and pass them as an argument to any relevant functions; this is the "owner" field of ntsync_wait_args and ntsync_mutex_args. When the emulator detects that a thread dies, it should therefore call NTSYNC_IOC_MUTEX_KILL on any open mutexes. * ntsync is module-capable mostly because there was nothing preventing it, and because it aided development. It is not a hard requirement, though. == Previous versions == Changes from the last (v2) RFC: * Add a new wait flag NTSYNC_WAIT_REALTIME. I had originally missed a corner case in NtWaitForMultipleObjects() related to its interaction with system time adjustments. Essentially the function is sometimes supposed to respect system time adjustments and sometimes supposed to ignore them, so in order to achieve this I've added a function that controls which flag is being synchronized to. Thanks Piotr Caban for catching this. * Add tests for overflowing semaphore and mutex counters, and a test for exceeding NTSYNC_MAX_WAIT_COUNT, per Andi Kleen. * Add a more intense and realistic test involving multiple threads using the same mutex to access data, per Andi Kleen. * Use check_add_overflow() instead of writing out overflow checking manually [and thereby avoid relying on -fwrapv]. * Add some missing headers that were being implicitly included: atomic.h, hrtimer.h, ktime.h, sched.h, sched/signal.h, spinlock.h. * Link to RFC v2: https://lore.kernel.org/lkml/20240131021356.10322-1-zfigura(a)codeweavers.c… * Link to RFC v1: https://lore.kernel.org/lkml/20240124004028.16826-1-zfigura(a)codeweavers.c… Elizabeth Figura (31): ntsync: Introduce the ntsync driver and character device. ntsync: Introduce NTSYNC_IOC_CREATE_SEM. ntsync: Introduce NTSYNC_IOC_SEM_POST. ntsync: Introduce NTSYNC_IOC_WAIT_ANY. ntsync: Introduce NTSYNC_IOC_WAIT_ALL. ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX. ntsync: Introduce NTSYNC_IOC_MUTEX_UNLOCK. ntsync: Introduce NTSYNC_IOC_MUTEX_KILL. ntsync: Introduce NTSYNC_IOC_CREATE_EVENT. ntsync: Introduce NTSYNC_IOC_EVENT_SET. ntsync: Introduce NTSYNC_IOC_EVENT_RESET. ntsync: Introduce NTSYNC_IOC_EVENT_PULSE. ntsync: Introduce NTSYNC_IOC_SEM_READ. ntsync: Introduce NTSYNC_IOC_MUTEX_READ. ntsync: Introduce NTSYNC_IOC_EVENT_READ. ntsync: Introduce alertable waits. ntsync: Allow waits to use the REALTIME clock. selftests: ntsync: Add some tests for semaphore state. selftests: ntsync: Add some tests for mutex state. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for manual-reset event state. selftests: ntsync: Add some tests for auto-reset event state. selftests: ntsync: Add some tests for wakeup signaling with events. selftests: ntsync: Add tests for alertable waits. selftests: ntsync: Add some tests for wakeup signaling via alerts. selftests: ntsync: Add a stress test for contended waits. maintainers: Add an entry for ntsync. docs: ntsync: Add documentation for the ntsync uAPI. Documentation/userspace-api/index.rst | 1 + .../userspace-api/ioctl/ioctl-number.rst | 2 + Documentation/userspace-api/ntsync.rst | 399 +++++ MAINTAINERS | 9 + drivers/misc/Kconfig | 9 + drivers/misc/Makefile | 1 + drivers/misc/ntsync.c | 1146 ++++++++++++++ include/uapi/linux/ntsync.h | 62 + tools/testing/selftests/Makefile | 1 + .../testing/selftests/drivers/ntsync/Makefile | 8 + tools/testing/selftests/drivers/ntsync/config | 1 + .../testing/selftests/drivers/ntsync/ntsync.c | 1407 +++++++++++++++++ 12 files changed, 3046 insertions(+) create mode 100644 Documentation/userspace-api/ntsync.rst create mode 100644 drivers/misc/ntsync.c create mode 100644 include/uapi/linux/ntsync.h create mode 100644 tools/testing/selftests/drivers/ntsync/Makefile create mode 100644 tools/testing/selftests/drivers/ntsync/config create mode 100644 tools/testing/selftests/drivers/ntsync/ntsync.c base-commit: e21817acb23ece75d41a4fa7b40c85550f147389 -- 2.43.0

Feb. 14, 2024

Wine staging 9.2 release

by Alistair Leslie-Hughes

Binary packages for various distributions will be available from: https://www.winehq.org/download Summary since last release * Rebased to current wine 9.2 (489 patches are applied to wine vanilla) Upstreamed (Either directly from staging or fixed with a similar patch). * d3dx9_36-D3DXDisassembleShader Added: * None Updated: * vkd3d-latest * ntdll-Junction_Points The patch count is down due to the upstreaming of the rawinput changes and due to conflicts, the wine-staging rawinput patches are disabled. Where can you help * Run Steam/Battle.net/GOG/UPlay/Epic * Test your favorite game. * Test your favorite applications. * Improve staging patches and get them accepted upstream. * Suggest patches to be included in staging. As always, if you find a bug, please report it via https://bugs.winehq.org Best Regards Alistair.

Feb. 11, 2024

Re: Wine staging 9.0 release

by Elizabeth Figura

On Sunday, 28 January 2024 11:07:28 CST DodoGTA GT wrote: > And finally on a unrelated note does the Compiler_Warnings patchset > bring any benefit to Wine? The compiler currently does not warn about > the implicit interface casts (and I don't think they're causing any > issues; removing that patchset could free up some work but that > patchset rarely breaks anyway so...) The meat of that patch set is patch 0031 ("include: Check element type in CONTAINING_RECORD and similar macros."). It was submitted once, and got a bit of pushback [1], but it's not clear to me whether it was enough pushback. I haven't tried to gauge whether it'd be acceptable now; I've been focusing on other more important patches first. [1] https://www.winehq.org/pipermail/wine-devel/2016-March/112260.html > > 2. I don't understand what you mean by this, can you please explain? > > You mentioned that in an answer to the question of wine-staging not > accepting MRs (merge requests/pull requests) when talking in the > modern version of IRC (aka Discord) so I want some clarification on > that (especially since you wrote a wall of text for question 4) I don't know what conversation you're referring to, but we don't take pull requests or patches because they are hard for contributors to get right, and even harder for us to review. You've kind of seen this by now, I hope. > > 3. The wine-staging team is documented here: > > https://wiki.winehq.org/Wine-Staging#What_is_Wine_Staging.3F > The wiki page mentions 2 people (Greer and Crider) I've never really > heard of before (are they inactive or are they working on different > parts of staging?) Thomas Crider joined at about the same time that Alistair and I picked up the project in 2018. He doesn't usually do rebases these days, but does often help with testing. Dean Greer has essentially built the wine-staging CI from scratch, and helped cover Mac testing and fixes, which as you might imagine tends to be an overlooked area. > > With respect, I need to see a demonstrated ability to contribute to the > > Wine project well, and a demonstrated ability to deal with the > > peculiarities Wine-Staging well, before considering anyone as an > > additional maintainer. This is not by any means a judgement on you or > > your ability; rather, as a very new contributor to the project, I simply > > haven't seen that ability demonstrated one way or the other. > So should I make my Wine work public and work actively on the project? That's preferable to working privately. But also, I wouldn't see "Wine-Staging maintainer" as a goal. Rather, I'd encourage you to consider contributing to Wine itself to be your end goal. Maintaining a project or subsystem is just a duty that goes along with that goal. > > I don't know what you mean by "hotfixes", not least because you're > > publishing them in some non-public place, so I can't really judge from > > that evidence either. > By "hotfixes" I meant making staging compatible with newer Wine > commits (aka rebasing) but these are quick and dirty (by that I mean > modifying the staging patches by hand) > > I mainly pushed these patches to a channel in a frog Discord server > (but no one likely used them anyway so I think I should publish them > to a public place like my wine-(dodo)staging repository on GitLab or > snippets/gists but I hope that the Robloxians don't accidentally find > them) Maybe I'm misunderstanding, but it sounds like you're duplicating the work of rebasing. I don't understand the point of this. Nothing depends on rebases getting done immediately. > And one more question that I've kind of forgotten about: Is there a > public archive of the bugs that were in bugs.wine-staging.com? Various > old Wine resources (and some staging patchset references) are useless > because of this (I think Alistair knows about this historical stuff > better so they should definitely look at this question) There is not unfortunately. If an archive exists, Michael or Sebastian has it, but I've tried to contact them asking for it and been unsuccessful. I'm afraid Alistair does not know any more than I do on this topic (unless I'm mistaken). --Zeb

Feb. 10, 2024

Re: Staging reparse point fix

by Elizabeth Figura

On Friday, 9 February 2024 11:18:34 CST DodoGTA GT wrote: > Hello, > > I have made an updated version of the reparse points patchset fix that > should reduce the patch noise quite a bit (I think the original > version is better in terms of code duplication though) Thanks, this makes it clearer what you're trying to fix. I've rewritten the fix for patch 5 to properly handle delayed closes. It's hard to read the diff but I suspect your original patch did that, and was more correct in that respect. Unfortunately reading diffs of diffs is very hard (even with practice), and reviewing them is an order of magnitude harder. In the future, if you have a change for wine-staging to submit, I'd ask that it either be in the form of a first-level diff against the Wine tree, which we'll apply as a separate patch or merge into the correct patches as appropriate, or simply a description of the change to be made. E.g. in this case "unlink() was incorrectly replaced with unlink_closed_fd(), when instead the diff to inode_destroy() should have been moved inside unlink_closed_fd()". I know the latter seems well against the usual practice of free software collaboration, but it really is easier for us to deal with. I've also removed the extraneous parts from your other patch, and applied the remaining parts to the relevant patches. Thanks for the patches. --Zeb

Feb. 9, 2024

Staging reparse point fix

by DodoGTA GT

Hello, I have made an updated version of the reparse points patchset fix that should reduce the patch noise quite a bit (I think the original version is better in terms of code duplication though)

Feb. 9, 2024

Re: kernelbase: adding support for callback in CopyFileEx, and implementing CopyFile2

by Kartavya Vashishtha

Adding CopyFile2 now has a MR, but I'm unsure of who I should request a review from or how to trigger the auto-assign reviewer functionality. https://gitlab.winehq.org/wine/wine/-/merge_requests/5020 Sincerely, Kartavya Vashishtha On Fri, 1 Dec 2023 at 14:46, Zeb Figura <zfigura(a)codeweavers.com> wrote: > On Friday, 1 December 2023 12:22:28 CST Nikolay Sivov wrote: > > > There seem to have been multiple efforts to implement this > functionality > > > (in 2022 and 2013) but I can't tell find any reasoning for why they > were > > > not successful. Would implementing this be a desirable contribution? > > > > > > > Sure, I recall we had a bug report for missing callbacks. > > That'd be bug 22690 and bug 22692. > > > > >

Feb. 8, 2024

Re: [RFC PATCH v2 19/29] selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY.

by Elizabeth Figura

On Wednesday, 31 January 2024 02:52:11 CST Andi Kleen wrote: > Elizabeth Figura <zfigura(a)codeweavers.com> writes: > > > +TEST(test_wait_any) > > +{ > > + struct ntsync_mutex_args mutex_args = {0}; > > + struct ntsync_wait_args wait_args = {0}; > > + struct ntsync_sem_args sem_args = {0}; > > + __u32 owner, index, count; > > + struct timespec timeout; > > + int objs[2], fd, ret; > > + > > + clock_gettime(CLOCK_MONOTONIC, &timeout); > > + > > + fd = open("/dev/ntsync", O_CLOEXEC | O_RDONLY); > > + ASSERT_LE(0, fd); > > + > > + sem_args.count = 2; > > + sem_args.max = 3; > > + sem_args.sem = 0xdeadbeef; > > + ret = ioctl(fd, NTSYNC_IOC_CREATE_SEM, &sem_args); > > + EXPECT_EQ(0, ret); > > + EXPECT_NE(0xdeadbeef, sem_args.sem); > > + > > + mutex_args.owner = 0; > > + mutex_args.count = 0; > > + mutex_args.mutex = 0xdeadbeef; > > + ret = ioctl(fd, NTSYNC_IOC_CREATE_MUTEX, &mutex_args); > > + EXPECT_EQ(0, ret); > > + EXPECT_NE(0xdeadbeef, mutex_args.mutex); > > It seems your tests are missing test cases for exceeding any limits, > especially overflow/underflow cases. Since these are the most likely > for any security problems it would be good to have extra coverage here. > The fuzzers will hopefully hit it too. > > Also some stress testing with multiple threads would be useful. Thanks, I'll add these.

Feb. 3, 2024

Upcoming vkd3d 1.11 release

by Henri Verbeet

Hi all, The tentative release date for vkd3d 1.11 is March 5. As usual, that means the development branch will become increasingly frozen as we're nearing release. In particular, the public API should be considered frozen after February 6, and we'll only accept targetted fixes after February 20. If you're an upstream vkd3d developer, please do the following: - Keep an eye on Bugzilla for any reported regressions. - Review any API changes since the previous release. - Verify all the vkd3d tests pass on your hardware, and please send fixes if they don't. - Review and improve the documentation. - Apply your best judgment about which patches are appropriate for the current stage of the release process. If you're a downstream user of vkd3d, please test your applications for regressions, and file bug reports if you find any. If you're the maintainer of a Wine module that (indirectly) depends on vkd3d, like e.g. d2d1, d3dx9, d3dcompiler, or any of the core Direct3D modules, that includes making sure the unit tests still pass with the new version. The planned release date for vkd3d 1.12 is currently May 28. Regards, Henri

Feb. 1, 2024