[PATCH 0/1] MR11198: Draft: winex11.drv/init: Use a depth-32 X visual for OpenGL layered windows.

[Jacob Czekalla (＠JacobCzekalla)]

When using WPF with hardware acceleration, layered windows can render incorrectly if they are created with the default depth-24 X visual.

-- 
https://gitlab.winehq.org/wine/wine/-/merge_requests/11198

[PATCH 1/1] winex11.drv/init: Use a depth-32 X visual for OpenGL layered windows.

[Jacob Czekalla]

From: Jacob Czekalla <[e-mail]>

When using WPF with hardware acceleration, layered windows can render incorrectly if they are created with the default depth-24 X visual.
---
 dlls/winex11.drv/init.c | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/dlls/winex11.drv/init.c b/dlls/winex11.drv/init.c
index d5b4fdbf0aa..260eeec8861 100644
--- a/dlls/winex11.drv/init.c
+++ b/dlls/winex11.drv/init.c
@@ -459,6 +459,32 @@ static int visual_class_alloc( int class )
     return class == PseudoColor || class == GrayScale || class == DirectColor ? AllocAll : AllocNone;
 }
 
+static BOOL find_depth32_visual( Display *display, int screen, XVisualInfo *out )
+{
+    XVisualInfo template = { 0 }, *info;
+    int count, i;
+
+    template.screen = screen;
+    template.depth = 32;
+    template.class = TrueColor;
+
+    info = XGetVisualInfo( display, VisualScreenMask | VisualDepthMask | VisualClassMask, &template, &count );
+    if (!info) return FALSE;
+
+    for (i = 0; i < count; i++)
+    {
+        if (info[i].red_mask == 0x00ff0000 && info[i].green_mask == 0x0000ff00 && info[i].blue_mask == 0x000000ff)
+        {
+            *out = info[i];
+            XFree( info );
+            return TRUE;
+        }
+    }
+
+    XFree( info );
+    return FALSE;
+}
+
 struct x11drv_client_surface *impl_from_client_surface( struct client_surface *client )
 {
     assert( client->funcs == &x11drv_client_surface_funcs );
@@ -470,8 +496,13 @@ struct client_surface *X11DRV_CreateClientSurface( HWND hwnd, int format )
     struct x11drv_client_surface *surface;
     XVisualInfo visual = default_visual;
     Colormap colormap;
+    DWORD xscreen, exstyle;
 
-    if (format && !visual_from_pixel_format( format, &visual )) return NULL;
+    exstyle = NtUserGetWindowLongW(hwnd, GWL_EXSTYLE);
+    xscreen = XScreenNumberOfScreen( visual.screen ? NULL : NULL );
+    if (use_egl && (exstyle & WS_EX_LAYERED) && find_depth32_visual( gdi_display, xscreen, &visual ))
+        TRACE( "Using 32-bit depth visual %#lx for window hwnd %p\n", visual.visualid, hwnd );
+    else if (format && !visual_from_pixel_format( format, &visual )) return NULL;
 
     if (visual.visualid == default_visual.visualid) colormap = default_colormap;
     else colormap = XCreateColormap( gdi_display, get_dummy_parent(), visual.visual, visual_class_alloc( visual.class ) );
-- 
GitLab

https://gitlab.winehq.org/wine/wine/-/merge_requests/11198

[Rémi Bernon (＠rbernon)]

Rémi Bernon (@rbernon) commented about dlls/winex11.drv/init.c:
> struct x11drv_client_surface *surface;
>      XVisualInfo visual = default_visual;
>      Colormap colormap;
> +    DWORD xscreen, exstyle;
>
> -    if (format && !visual_from_pixel_format( format, &visual )) return NULL;
> +    exstyle = NtUserGetWindowLongW(hwnd, GWL_EXSTYLE);
> +    xscreen = XScreenNumberOfScreen( visual.screen ? NULL : NULL );
> +    if (use_egl && (exstyle & WS_EX_LAYERED) && find_depth32_visual( gdi_display, xscreen, &visual ))
> +        TRACE( "Using 32-bit depth visual %#lx for window hwnd %p\n", visual.visualid, hwnd );
> +    else if (format && !visual_from_pixel_format( format, &visual )) return NULL;

There's a `argb_visual` global variable defined, that layered windows usually directly use, it may be what we need to use.

It's also possible that this doesn't belong there and that the pixel format, which pretty much drives the visual selection here, should be tweaked to only allow the correct visuals for layered windows. We could have tests to check whether layered windows have a different set of pixel formats for instance. Then even if that's not the case I'm not sure selecting a different visual than what the pixel format describes is going to work either.

-- 
https://gitlab.winehq.org/wine/wine/-/merge_requests/11198#note_143661