From a time till now, I am using a test application in Visual Basic in order to test progress in quartz. The code for the application (along with a precompiled EXE) can be found at the following address:
http://www.palosanto.com/~a_villacis/WMediaTest.tar.bz2
One of the things I dislike about the current quartz implementation is that when a large AVI video file (say, 640x480) is played with the test application, the sound stutters (in both ALSA and OSS) while the frames are being updated. This occurs with the builtin quartz.dll, but not with native (WinXP) quartz. From what I can deduce from WINEDEBUG=+tid,+quartz , it seems that builtin feeds video and audio in the same thread. In addition, dlls/quartz/videorenderer.c uses StretchDIBits() in order to write one frame of video into the output window. Apparently StretchDIBits() is slow enough in my machine that it cannot complete by the next sound sample. Therefore the stuttering.
I made a little patch to move the video rendering in videorender.c into a separate thread, in order to allow it to drop frames. However this does not completely eliminate the stuttering audio. I am now thinking about rewriting the video rendering with DirectDraw. It seems that native quartz.dll uses DirectDraw, not GDI, to display video frames into the output window.
One thing I am worried about is the pixel format of the surfaces. The video data is 32bpp, but the output window uses my X config at 16bpp. However, nowhere in the ddraw trace of native quartz I can find any reference to a CreateSurface with 32bpp. It seems that native quartz does the depth conversion itself. The question is this: if I create a 32 bpp surface (in order to lock it and write the 32 bpp frame data into it), can I then blit it into a 16 bpp output window and somehow let DirectDraw take care of the depth conversion? If so, can this be any faster than writing the depth conversion myself?
Alex Villacís Lasso
Hi,
I am now thinking about rewriting the video rendering with DirectDraw. It seems that native quartz.dll uses DirectDraw, not GDI, to display video frames into the output window.
DirectDraw sounds like the right way to implement quartz video output, but performance wise I am concerned about the performance of our current directdraw implementation. DirectDraw is implemented in software, on top of gdi, so you end up with just one more abstraction layer in between. Our DDraw code basically has the capability to use opengl for hardware acceleration, but its far from complete.
One thing I am worried about is the pixel format of the surfaces. The video data is 32bpp, but the output window uses my X config at 16bpp. However, nowhere in the ddraw trace of native quartz I can find any reference to a CreateSurface with 32bpp. It seems that native quartz does the depth conversion itself. The question is this: if I create a 32 bpp surface (in order to lock it and write the 32 bpp frame data into it), can I then blit it into a 16 bpp output window and somehow let DirectDraw take care of the depth conversion? If so, can this be any faster than writing the depth conversion myself?
You cannot create a 16 bit front buffer on a 32 bit desktop, except if you switch to exclusive mode(and thus fullscreen). The really proper way to handle this is to create an overlay surface in your video's input format and overlay the primary(= the screen) with it. The issue with this is that wine does not support overlay surfaces yet. There is an overlay implementation via Xv in CodeWeaver's Picasa wine tree(downloadable at www.google.com). I was about to integrate it into wine, but some problems with accessing Xv from wined3d arised, and the overall idea was to wait for the wined3d using WGL instead of GLX efforts, which somehow froze with the windowed GL rendering problems.
The other idea would be to create a Direct3D device(with the back buffer beeing automatically in desktop bit depth), + a few(dynamic) textures in the video's format and size. Then upload the video into the textures and draw quads on the back buffer, then flip. This way the opengl hardware does the color conversion and filtering for you. The drawback is that you need hardware acceleration.
Obviously, if you can use Direct3D, and Direct3D uses OpenGL, you could use OpenGL directly, which saves you a not really cheap layer and gets you more control over opengl. The advantage of using Direct3D would be that an Overlay path and a Direct3D path could share some code, but I don't know how much.
On 13/06/07, Stefan Dösinger stefandoesinger@gmx.at wrote:
Obviously, if you can use Direct3D, and Direct3D uses OpenGL, you could use OpenGL directly, which saves you a not really cheap layer and gets you more control over opengl. The advantage of using Direct3D would be that an Overlay path and a Direct3D path could share some code, but I don't know how much.
It would require you to setup a GL context though.
Stefan Dösinger escribió:
Hi,
I am now thinking about rewriting the video rendering with DirectDraw. It seems that native quartz.dll uses DirectDraw, not GDI, to display video frames into the output window.
DirectDraw sounds like the right way to implement quartz video output, but performance wise I am concerned about the performance of our current directdraw implementation. DirectDraw is implemented in software, on top of gdi, so you end up with just one more abstraction layer in between. Our DDraw code basically has the capability to use opengl for hardware acceleration, but its far from complete.
One thing I am worried about is the pixel format of the surfaces. The video data is 32bpp, but the output window uses my X config at 16bpp. However, nowhere in the ddraw trace of native quartz I can find any reference to a CreateSurface with 32bpp. It seems that native quartz does the depth conversion itself. The question is this: if I create a 32 bpp surface (in order to lock it and write the 32 bpp frame data into it), can I then blit it into a 16 bpp output window and somehow let DirectDraw take care of the depth conversion? If so, can this be any faster than writing the depth conversion myself?
You cannot create a 16 bit front buffer on a 32 bit desktop, except if you switch to exclusive mode(and thus fullscreen). The really proper way to handle this is to create an overlay surface in your video's input format and overlay the primary(= the screen) with it. The issue with this is that wine does not support overlay surfaces yet. There is an overlay implementation via Xv in CodeWeaver's Picasa wine tree(downloadable at www.google.com). I was about to integrate it into wine, but some problems with accessing Xv from wined3d arised, and the overall idea was to wait for the wined3d using WGL instead of GLX efforts, which somehow froze with the windowed GL rendering problems.
The other idea would be to create a Direct3D device(with the back buffer beeing automatically in desktop bit depth), + a few(dynamic) textures in the video's format and size. Then upload the video into the textures and draw quads on the back buffer, then flip. This way the opengl hardware does the color conversion and filtering for you. The drawback is that you need hardware acceleration.
Obviously, if you can use Direct3D, and Direct3D uses OpenGL, you could use OpenGL directly, which saves you a not really cheap layer and gets you more control over opengl. The advantage of using Direct3D would be that an Overlay path and a Direct3D path could share some code, but I don't know how much.
Maybe I forgot to explain. My test application was run with WINEDLLOVERRIDES=quartz=n in order to use the native quartz.dll . With this, audio does not stutter. The explanation about the DirectDraw implementation in Wine being based on top of GDI does not explain how native quartz manages to play smoothly while builtin quartz stutters *when both are run under Wine*. Maybe the real trick is that native quartz does the 32bpp -> 16 bpp conversion in a way that is faster than the one made by StretchDIBits(). If this is correct, then I should implement my own depth conversion as part of the rewrite and then use DirectDraw to blit the resulting frame into the output. What do you think?
On Tuesday 12 June 2007 04:36:52 pm Alex Villacís Lasso wrote:
The explanation about the DirectDraw implementation in Wine being based on top of GDI does not explain how native quartz manages to play smoothly while builtin quartz stutters *when both are run under Wine*.
Because built-in quartz uses GDI, which is slow as fsck. Native I'm quite sure uses D3D, which in Wine will go through OpenGL and still be accelerated. OpenGL will do stretching and color conversions in hardware so it's not so slow.
Maybe I forgot to explain. My test application was run with WINEDLLOVERRIDES=quartz=n in order to use the native quartz.dll . With this, audio does not stutter. The explanation about the DirectDraw implementation in Wine being based on top of GDI does not explain how native quartz manages to play smoothly while builtin quartz stutters *when both are run under Wine*.
Yes, I understood that part in your first mail. My point was that while the implementation of native quartz MAY be more efficient, using DirectDraw in Wine is not faster than using GDI directly per se. That doesn't mean that native can't be faster than builtin, but that would be because of a better implementation, not because it uses 2D ddraw. Direct3D, on the other hand, would be hw accelerated, and thus faster. Sorry that I didn't make this clear.
-
Alex Villacís Lasso -
Chris Robinson -
H. Verbeet -
Stefan Dösinger