[RFC] Announcing Wayland driver development
Hi all, For some time now I have been working on a Wayland driver for Wine, and it has now reached a good enough state to present it to the wider community. This driver allows users to run Windows GDI and OpenGL applications directly on Wayland compositors without an intermediate layer to translate from X11 to Wayland. This leads to a leaner and more efficient stack. The code currently resides at: https://gitlab.collabora.com/alf/wine/-/commits/wayland Get it with: $ git clone -b wayland https://gitlab.collabora.com/alf/wine/ Configure with (you need the wayland-client and wayland-egl packages): $ ./configure --with-wayland To run, ensure there is a Wayland compositor running, unset DISPLAY (so that Wayland is picked instead of X11) and set WAYLAND_DISPLAY (if needed): $ DISPLAY= WAYLAND_DISPLAY=wayland-0 ./wine ... Here is video showcasing a few Windows applications running with the Wayland driver on the Weston reference compositor: https://youtu.be/br2b8gUy5n8 The Wayland protocol is by design more constrained compared to more traditional display systems like X11. These constraints add some special challenges in the integration of Wayland with Win32. Since Wayland's window model is not based on a single flat 2D co-ordinate space, as X11's was, the Wayland protocol doesn't allow applications to control their absolute position on the screen. For Win32 transient windows (menus, tooltips, etc) the driver tries to work around the lack of absolute positioning by "anchoring" them to an owning Wayland surface and treating them as subsurfaces of that owner. Screen coordinates for such windows are transformed to local coordinates relative to the owning surface, allowing correct placement through relative subsurface movement, which is supported by Wayland. By using heuristics to select the proper owning surface, this approach has led to very good results. Absolute positioning of non-transient top level windows is not supported at this time, and will likely require a (possibly controversial) Wayland extension if it is to ever be pursued. The lack of absolute positioning also has an adverse effect on input handling, when parts of windows reside outside the visible windows display space, but are still full accessible in the Wayland compositor display space (and thus the user). Input events to such areas don't reach their intended coordinates (are clamped to Windows display bounds) leaving the user mystified about why they are unable to interact with a perfectly visible area in their Wayland compositor. The current partial workaround is to force all windows at (0,0) windows display coordinates, to maximize the area which can be interacted with. With this workaround, and as long as windows remain smaller than the display size, input works without issues. Here what's supported at the moment (modulo bugs): * GDI apps, including layered/translucent windows * OpenGL apps * Window resizing * Maximized and fullscreen window states * Mouse and keyboard (only QWERTY) input * Mouse cursors * Menus, tooltips, etc * Single display What needs more work (or is completely missing): * Minimize * Better z-order and activation handling * Keyboard layout support * Grabs * GTK menu mouse handling bug (cannot click to select items) * Multiple displays * Vulkan. Note that that there is another effort at https://github.com/varmd/wine-wayland/ focusing solely on vulkan. My hope is that we will be able to share efforts going forward. Programs I have tried and which are working well (again modulo bugs): * Native Wine/Win32 apps (notepad, regedit etc) * Supertuxkart * 010Editor * Firefox * Stellarium * Battle For Wesnoth * GIMP (except clicking items in GTK menus as noted above) * Unigine Valley My first question to the Wine community is how complete the driver should be before being considered for inclusion. Is the current Wayland driver state enough? If there is interest, I was hoping to get the driver in earlier rather than later, in an experimental capacity, in order to provide a single, official point of development for people wanting to contribute. My second question is about the best way to move forward with upstreaming. The code is currently split between just a few commits, mostly as a result of the experimental nature of this effort, and also because I used existing code from winex11 and wineandroid as my starting point. I would like to split this into smaller commits to make it more review-friendly for the mailing list. I was thinking of splitting by user32 function (and internal dependencies), but since this is going to be an artificial split, I am open to other ideas to make reviewers' lives easier. Looking forward to your feedback and questions! Thanks, Alexandros
This is super amazing. Thank you very much for this contribution to WINE! I can't wait to try this. Cheers, Aaryaman On Wed, Dec 16, 2020 at 2:16 AM Alexandros Frantzis < alexandros.frantzis(a)collabora.com> wrote:
Hi all,
For some time now I have been working on a Wayland driver for Wine, and it has now reached a good enough state to present it to the wider community. This driver allows users to run Windows GDI and OpenGL applications directly on Wayland compositors without an intermediate layer to translate from X11 to Wayland. This leads to a leaner and more efficient stack.
The code currently resides at:
https://gitlab.collabora.com/alf/wine/-/commits/wayland
Get it with:
$ git clone -b wayland https://gitlab.collabora.com/alf/wine/
Configure with (you need the wayland-client and wayland-egl packages):
$ ./configure --with-wayland
To run, ensure there is a Wayland compositor running, unset DISPLAY (so that Wayland is picked instead of X11) and set WAYLAND_DISPLAY (if needed):
$ DISPLAY= WAYLAND_DISPLAY=wayland-0 ./wine ...
Here is video showcasing a few Windows applications running with the Wayland driver on the Weston reference compositor:
The Wayland protocol is by design more constrained compared to more traditional display systems like X11. These constraints add some special challenges in the integration of Wayland with Win32.
Since Wayland's window model is not based on a single flat 2D co-ordinate space, as X11's was, the Wayland protocol doesn't allow applications to control their absolute position on the screen. For Win32 transient windows (menus, tooltips, etc) the driver tries to work around the lack of absolute positioning by "anchoring" them to an owning Wayland surface and treating them as subsurfaces of that owner. Screen coordinates for such windows are transformed to local coordinates relative to the owning surface, allowing correct placement through relative subsurface movement, which is supported by Wayland. By using heuristics to select the proper owning surface, this approach has led to very good results.
Absolute positioning of non-transient top level windows is not supported at this time, and will likely require a (possibly controversial) Wayland extension if it is to ever be pursued. The lack of absolute positioning also has an adverse effect on input handling, when parts of windows reside outside the visible windows display space, but are still full accessible in the Wayland compositor display space (and thus the user). Input events to such areas don't reach their intended coordinates (are clamped to Windows display bounds) leaving the user mystified about why they are unable to interact with a perfectly visible area in their Wayland compositor. The current partial workaround is to force all windows at (0,0) windows display coordinates, to maximize the area which can be interacted with. With this workaround, and as long as windows remain smaller than the display size, input works without issues.
Here what's supported at the moment (modulo bugs):
* GDI apps, including layered/translucent windows * OpenGL apps * Window resizing * Maximized and fullscreen window states * Mouse and keyboard (only QWERTY) input * Mouse cursors * Menus, tooltips, etc * Single display
What needs more work (or is completely missing):
* Minimize * Better z-order and activation handling * Keyboard layout support * Grabs * GTK menu mouse handling bug (cannot click to select items) * Multiple displays * Vulkan. Note that that there is another effort at https://github.com/varmd/wine-wayland/ focusing solely on vulkan. My hope is that we will be able to share efforts going forward.
Programs I have tried and which are working well (again modulo bugs):
* Native Wine/Win32 apps (notepad, regedit etc) * Supertuxkart * 010Editor * Firefox * Stellarium * Battle For Wesnoth * GIMP (except clicking items in GTK menus as noted above) * Unigine Valley
My first question to the Wine community is how complete the driver should be before being considered for inclusion. Is the current Wayland driver state enough? If there is interest, I was hoping to get the driver in earlier rather than later, in an experimental capacity, in order to provide a single, official point of development for people wanting to contribute.
My second question is about the best way to move forward with upstreaming. The code is currently split between just a few commits, mostly as a result of the experimental nature of this effort, and also because I used existing code from winex11 and wineandroid as my starting point. I would like to split this into smaller commits to make it more review-friendly for the mailing list. I was thinking of splitting by user32 function (and internal dependencies), but since this is going to be an artificial split, I am open to other ideas to make reviewers' lives easier.
Looking forward to your feedback and questions!
Thanks, Alexandros
On Tue, Dec 15, 2020 at 7:16 PM Alexandros Frantzis < alexandros.frantzis(a)collabora.com> wrote:
What needs more work (or is completely missing):
* Minimize * Better z-order and activation handling * Keyboard layout support * Grabs * GTK menu mouse handling bug (cannot click to select items) * Multiple displays * Vulkan. Note that that there is another effort at https://github.com/varmd/wine-wayland/ focusing solely on vulkan. My hope is that we will be able to share efforts going forward.
That's impressive. What about: Clipboard Drag and drop Cross-process window embedding / system tray X11-based XDG standards, eg. startup notifications Games that change screen resolution / color depth Damjan
On Wed, Dec 16, 2020 at 06:35:48AM +0200, Damjan Jovanovic wrote:
On Tue, Dec 15, 2020 at 7:16 PM Alexandros Frantzis < alexandros.frantzis(a)collabora.com> wrote:
What needs more work (or is completely missing):
* Minimize * Better z-order and activation handling * Keyboard layout support * Grabs * GTK menu mouse handling bug (cannot click to select items) * Multiple displays * Vulkan. Note that that there is another effort at https://github.com/varmd/wine-wayland/ focusing solely on vulkan. My hope is that we will be able to share efforts going forward.
Hi!
What about: Clipboard Drag and drop Games that change screen resolution / color depth
These are all in my TODO list (I forgot to mention them in the original list!). Note that Wayland doesn't allow clients to change the monitor display mode per se, but we can still get the desired effect by providing to the game its desired "fullscreen" size window, and instructing the Wayland compositor to scale the window to match the size of the real current monitor mode.
Cross-process window embedding / system tray
I haven't looked into this yet. I'll need to investigate what the requirements are from the Windows/Wine side, and whether relevant Wayland extensions ("foreign surfaces") can accommodate those requirements.
X11-based XDG standards, eg. startup notifications
I haven't considered support for the XDG standards yet, and we will have to examine each standard we care about separately, to see what Wayland supports or could reasonably support. For startup notifications in particular, Wayland doesn't currently provide a standard extension, but it has been discussed. For now, GTK/gnome provides relevant functionality through their own gtk-shell1 extension. One way forward, and until more standard support is provided, is to use this extension (if present) to at least get startup notifications on gnome-shell. Thanks, Alexandros
Hi Alexandros, first of all, congratulations and thanks for your work, this sounds very cool! Disclaimer: I'm little more than a curious observer of the larger developments in Wine, but I've been around for a while and have picked up a few things. Still, you should probably take anything someone who actually understands the code says over my ramblings ;)
My first question to the Wine community is how complete the driver should be before being considered for inclusion. Is the current Wayland driver state enough?
The most important and probably trickiest order of business is to convince Alexandre you're going the "Right Way" (tm), from an architectural standpoint. I can't really say if your current progress is enough to gauge that, but I think it is.
If there is interest, I was hoping to get the driver in earlier rather than later, in an experimental capacity, in order to provide a single, official point of development for people wanting to contribute.
You should talk to our tireless and competent staging developers, Zebediah Figura and Alistair Leslie-Hughes about that.
My second question is about the best way to move forward with upstreaming. The code is currently split between just a few commits, mostly as a result of the experimental nature of this effort, and also because I used existing code from winex11 and wineandroid as my starting point. I would like to split this into smaller commits to make it more review-friendly for the mailing list. I was thinking of splitting by user32 function (and internal dependencies), but since this is going to be an artificial split, I am open to other ideas to make reviewers' lives easier.
Again, I'm no expert, but from what I've seen, a good approach would be to start with a stubbed skeleton driver and build it up from that, piece by piece, in small logical steps. The rules for commits are basically: * leave wine in a consistent and working state after every commit! * don't introduce dead code * have meaningful commit messages (* normally tests are very important, though I'm not sure how much that applies in this case or how to go about failing tests under your driver) Read the wiki for more info, notably https://wiki.winehq.org/Submitting_Patches and most of https://wiki.winehq.org/Wine_Developer%27s_Guide though some stuff may be out of date. FWIW, I recommend looking at the commit history, how past big changes were introduced i.e. the Mac driver, maybe the DIB-Engine (although that was quite long ago and a lot has changed since) Better wait for advice from actual developers here ;)
Looking forward to your feedback and questions!
I hope that helps a bit, and that when the experts ship in they don't have to repeat the basics. This is about all I know, sorry I can't really be more specific. Please correct anything I got wrong! Best, Julian
FWIW, I recommend looking at the commit history, how past big changes were introduced i.e. the Mac driver, maybe the DIB-Engine (although that was quite long ago and a lot has changed since)
The most recent example of a new driver: https://source.winehq.org/git/wine.git/?a=search&h=HEAD&st=commit&s=wineusb
On Wed, Dec 16, 2020 at 10:24:43AM +0100, Julian Rüger wrote:
FWIW, I recommend looking at the commit history, how past big changes were introduced i.e. the Mac driver, maybe the DIB-Engine (although that was quite long ago and a lot has changed since)
The most recent example of a new driver:
https://source.winehq.org/git/wine.git/?a=search&h=HEAD&st=commit&s=wineusb
Hi Julian, I really appreciate all the feedback and pointers! Thanks, Alexandros
On 12/16/20 2:44 AM, Julian Rüger wrote:
Hi Alexandros,
first of all, congratulations and thanks for your work, this sounds very cool!
Disclaimer:
I'm little more than a curious observer of the larger developments in Wine, but I've been around for a while and have picked up a few things. Still, you should probably take anything someone who actually understands the code says over my ramblings ;)
My first question to the Wine community is how complete the driver should be before being considered for inclusion. Is the current Wayland driver state enough?
The most important and probably trickiest order of business is to convince Alexandre you're going the "Right Way" (tm), from an architectural standpoint. I can't really say if your current progress is enough to gauge that, but I think it is.
If there is interest, I was hoping to get the driver in earlier rather than later, in an experimental capacity, in order to provide a single, official point of development for people wanting to contribute.
You should talk to our tireless and competent staging developers, Zebediah Figura and Alistair Leslie-Hughes about that.
That's probably about right. I would like to be sure that a Wayland driver will be accepted upstream in some form or another before we take it—I don't want to maintain any patch set in wine-staging forever, but an entire graphics driver least of all. In particular I've seen a lot of concerns about the long list of things Wayland can't support. Window positioning is one, but not the only by far. See also [1] (not that I can personally verify the veracity of any of these; I don't work with window managers). [1] https://bugs.winehq.org/show_bug.cgi?id=42284#c17
My second question is about the best way to move forward with upstreaming. The code is currently split between just a few commits, mostly as a result of the experimental nature of this effort, and also because I used existing code from winex11 and wineandroid as my starting point. I would like to split this into smaller commits to make it more review-friendly for the mailing list. I was thinking of splitting by user32 function (and internal dependencies), but since this is going to be an artificial split, I am open to other ideas to make reviewers' lives easier.
Again, I'm no expert, but from what I've seen, a good approach would be to start with a stubbed skeleton driver and build it up from that, piece by piece, in small logical steps.
The rules for commits are basically: * leave wine in a consistent and working state after every commit! * don't introduce dead code * have meaningful commit messages
(* normally tests are very important, though I'm not sure how much that applies in this case or how to go about failing tests under your driver)
Read the wiki for more info, notably https://wiki.winehq.org/Submitting_Patches and most of https://wiki.winehq.org/Wine_Developer%27s_Guide though some stuff may be out of date.
FWIW, I recommend looking at the commit history, how past big changes were introduced i.e. the Mac driver, maybe the DIB-Engine (although that was quite long ago and a lot has changed since)
Better wait for advice from actual developers here ;)
I think you've described our development guidelines pretty well ;-) It's probably worth pointing out that "leave wine in a working state" is a bit less important when it comes to things like introducing a new driver, at least with respect to that driver itself. You definitely shouldn't break anything when Wayland isn't used, and ideally something working in Wayland from an earlier patch shouldn't be broken by a later one (e.g. the style of development that produces lots of "fixup earlier patch" patches is frowned upon), but of course it's expected that the driver won't be working on Wayland in every patch.
Looking forward to your feedback and questions!
I hope that helps a bit, and that when the experts ship in they don't have to repeat the basics.
This is about all I know, sorry I can't really be more specific. Please correct anything I got wrong!
Best, Julian
On Wed, Dec 16, 2020 at 10:07:09AM -0600, Zebediah Figura (she/her) wrote:
On 12/16/20 2:44 AM, Julian Rüger wrote: <snip>
You should talk to our tireless and competent staging developers, Zebediah Figura and Alistair Leslie-Hughes about that.
That's probably about right. I would like to be sure that a Wayland driver will be accepted upstream in some form or another before we take it—I don't want to maintain any patch set in wine-staging forever, but an entire graphics driver least of all.
In particular I've seen a lot of concerns about the long list of things Wayland can't support. Window positioning is one, but not the only by far. See also [1] (not that I can personally verify the veracity of any of these; I don't work with window managers).
Hi Zebediah, Thank you for your comments and the link to the bug! I would like to use this opportunity to address some of the points mentioned in the bug report, since quite a few people may also have similar concerns. There is no question that the direct Wine->X11 path is likely to remain the most capable for the foreseeable future. However, since Wayland adoption is growing, and, by some accounts at least, X11 is becoming legacy, I think it's worth it for Wine to begin exploring the Wayland path. It won't be perfect in the beginning, although it may be "good enough" for most people, and given the special weight that Wine has, it may also help incentivize the implementation of missing Wayland functionality in the form of new extensions. Direct support for Wayland also provides access to modern features that may not be available to X11 clients. An example is HDR support, which is currently being worked on for Wayland [1], but unlikely to be supported on X11. --- Regarding some specific concerns:
global keybindings, clipboard, layout switching, screen capturing, tray icons,
With the exception of clipboard operations, for which Wayland has an official protocol, Wayland doesn't directly support the rest. However, a feature not being supported by Wayland doesn't mean much about its availability on a Wayland based system. Some features which were previously X11 requests, are now implemented using a completely different mechanism, mostly due to security considerations. For example, screen capturing is already supported through the xdg-portal mechanism on KDE, GNOME and wlroots based desktops. There are also discussions about implementing global key bindings in a similar way, or potentially as a Wayland protocol. --- I think some notes on XWayland are also warranted, since some may consider it to be a the way forward for Wine under Wayland. XWayland is privileged in many ways, being closely integrated with Wayland compositors. A Wayland compositor acts as the X11 window manager, which is how absolute window positioning from X11 can work on Wayland. Despite its special status, XWayland is still not given free rein over the compositor. There are many features which do not work through the X11 protocol on XWayland, and are likely to remain that way. Here is a list of such features that may be of interest to Wine: * screenshots, screencasting (except of other X11 windows) * global hotkeys * input grabs and eavesdropping * changing global state like "video card LUT" or video mode * controlling or inspecting other applications' windows (except possibly for other X11 apps) Some of these features will need to be accessed through the same alternative methods that one would need to use on direct Wayland too, some are waiting to be implemented with a Wayland protocol or other method, others are unlikely to ever be permitted on normal desktop systems. Due to the above, my opinion is that a direct Wayland driver is a much better long-term option for running Wine on Wayland. Thanks, Alexandros [1] https://www.collabora.com/news-and-blog/blog/2020/11/19/developing-wayland-c...
participants (5)
-
Aaryaman Vasishta -
Alexandros Frantzis -
Damjan Jovanovic -
Julian Rüger -
Zebediah Figura (she/her)