OpenGL in child windows

[Chris Robinson]

Here's something I've been working on for about the past week. It's my attempt 
to get OpenGL to render into an X11 child window that overlays the Win32 
window. These patches are more proof-of-concept than anything I'd attempt to 
get into Wine.. just to show that it seems to work. If the method is good 
enough, these would need to be rewritten.

The biggest problem facing these patches that I ran into are threading issues. 
Since each window is basically tied to the thread that it was created in but 
can be used out of it, and each thread uses its own X11 display connection, 
there's quite a few hoops to jump through.

The problem is that the OpenGL X11 window can be created at basically any time 
after the window was created, in any thread (it's currently done in the call 
to SetPixelFormat, the earliest it'll know which visual/pixel format is 
wanted, and can create a colormap to be able to use it in a child window, 
regardless of the visual of its parent). If that call is not done in the 
thread the win32 window was created in, it would have trouble getting some of 
the data structures it needs. Add on top of that, that the X11 window can be 
parented to yet another win32 window (the one that has the "whole" X11 
window), which can be in yet another thread, it leaves a lot of problem 
trying to keep syncronization.

Most of the placement syncronization is done by adding a couple fields to 
x11drv_win_data to hold info for the X11 child window's parent, that can be 
used to properly offset the X11 child in relation to the X11 parent, 
according to the Win32 window's placement. It also needed a couple 
driver-specific WM_ messages to be able to get the data it needs in case the 
creation is done in a seperate thread.

Because the X11 window uses a thread-specific display connection, and the 
child window needs to share the display connection, I needed to add a 
Display* to the X11DRV_PDEVICE struct, which is used wherever a GLX call 
needs an explicit display connection. This works well most of the time, but 
has a problem when a GL context (tied to an HDC in Windows) is in use when 
the thread that created the Win32 window that the HDC came from is killed. 
When the thread is killed, it closes the connection which leaves GLX using an 
invalid Display.

I have to ask, is there a reason each thread uses its own display connection 
(which are all different from gdi_display), and leaves the window data to 
only be available from the thread the window was created in? Besides the 
aforementioned problem with the GL context, I'd imagine it would be 
problematic even in current Wine.. for example, if something creates a window 
in one thread, passes it to another, then kills the thread it was originally 
created in while still using the window.

I currently made it so each thread shares the same Display connection (which 
is different from gdi_display). This seems to cause no noticeable problems, 
and it fixes the GL context display issue.

Beyond that, the only problem I can recall off-hand is that the OpenGL window 
will draw overtop of all Win32 windows that are sharing the same X11 parent 
window. However, considering that the current method with the scissors hack 
causes the whole X11 window to clear on a buffer flip, and also draws on top 
of anything in its window space anyway, this is still a nice step forward. 
And there are also a couple syslevel potential-deadlock warnings generated 
from the calls to create the window, however I have yet to see a problem 
arise from all the programs I've tested (the warnings can't stay obviously, 
but they can be figured out later).


What these patches end up accomplishing is that it allows for the possibility 
to use alternate visuals, so the X11 driver would not need to force a 
specific double-buffered visual on initialization, and it allows programs to 
use program-specified color, stencil, depth, aux, multisampling, and 
single/double-buffer configurations without relying on the main window to 
have it too (this isn't done in these patches, but it does allow for it to 
work). Also, the scissors hack is no longer needed. All calls to the 
Wine-specific extension for it can be removed (they aren't removed here, 
instead just piped directly to the appropriate gl* function with no 
alteration, since that would require changes to opengl32). OpenGL in Win32 
child windows is improved by being restrained to the Win32 windows' extents, 
as seen in the screen-grab here in VLC using the OpenGL output renderer:

http://kcat.strangesoft.net/deskshot.jpg

It uses proper color ordering (unlike the GDI renderer), is properly centered, 
can be scaled with no performance impact, and doesn't clear the whole window. 
Performance is good, and it is actually quite useable.

Terragen (which has a real-time 3D window), Terragen 2 (which has two or three 
OpenGL child windows), Milkshape 3D, and GoogleEarth all work. Terragen 2 has 
some issues with its right-click menu that comes up from the OpenGL child 
window (the mouse doesn't interact with it), however I'm not sure if this is 
an issue with my patches since Milkshape3D also has a right-click menu from 
an OpenGL child and it works fine. In any case, there's no other 
input-related issues to speak of. Clicking and dragging the globe in  
GoogleEarth works, as does the keyboard controls in Terragen, and the other 
Win32 windows are properly receptive to all input as well. It also behaves 
properly with composite effects (such as the transparency, stretching, and 
warping effects from Beryl).


All testing and comments on the patches are welcome. If the code is doing 
something it shouldn't be, I'm more than willing to figure out alternative 
methods if needed (hopefully with help). With luck, they can be gotten to a 
point soon where they can be rewritten properly to get into Wine.

[L. Rahyen]

Wednesday January 10 2007 06:27、Chris Robinson さんは書きました:
> Here's something I've been working on for about the past week. It's my
> attempt to get OpenGL to render into an X11 child window that overlays the
> Win32 window.

Your patches works perfectly! I have tested few applications with OpenGL 
windows and everything works perfectly. For better testing I have tested all 
Windows application I have including games to make sure there is no any 
regression. Everything works great including performance (i.e. I didn't find 
any bugs and 3D performance is as good as without patches). Well done!
> With luck, they can be gotten to a point soon where they can be rewritten
> properly to get into Wine.

Hope so because it's very important to get this in main tree before 1.0 
release. Now with your patchs all 3D professional software (which unusable 
because of OpenGL child windows in vanilla Wine) can work perfectly under 
Wine (Lightwave for example which has multiple OpenGL child windows works 
great).

[Frank Richter]

On 10.01.2007 07:27, Chris Robinson wrote:
> Beyond that, the only problem I can recall off-hand is that the OpenGL window 
> will draw overtop of all Win32 windows that are sharing the same X11 parent 
> window.

Hmm, if OGL is displayed in its own child window, maybe it's possible to
shape it so that overlapping Win32 child windows are "excluded"...

-f.r.

[Chris Robinson]

On Wednesday 10 January 2007 12:22, Frank Richter wrote:
> Hmm, if OGL is displayed in its own child window, maybe it's possible to
> shape it so that overlapping Win32 child windows are "excluded"...

I'm not sure how to do that, given that the overlaps can cause the child 
window to have concave shapes that change when the user moves child windows 
around.

One idea I had though, was to create another child window on the opengl child, 
and draw child windows that are above the opengl child to that (in addition 
to the "base" window), if it intersects. Would be harsh on performance, but 
ony when its intersecting.

Another idea was to somehow put the opengl window behind the parent window, 
and draw some special colorkey to "cut out" the X11 window and show the 
OpenGL child underneath. Sort of like an Xv overlay. That way, when win32 
windows draw on top of the win32 opengl child, the colorkey won't be applied, 
and the x11 child will be hidden. I'm just not sure if that's possible.

Short of that, I've thought about using an actual Xv overlay, but I can't find 
anywhere to get a drawable to redirect OpenGL to. Seems the only way to draw 
to such an overlay is with XPutImage (and redirecting OpenGL to a pixmap has 
speed issues, in addition to updating issues).

> I have to ask, is there a reason each thread uses its own display connection
> (which are all different from gdi_display), and leaves the window data to
> only be available from the thread the window was created in? Besides the
> aforementioned problem with the GL context, I'd imagine it would be
> problematic even in current Wine.. for example, if something creates a window
> in one thread, passes it to another, then kills the thread it was originally
> created in while still using the window.

In MS Windows, windows die if the creating thread exits for any reason.