[Bug 38196] New: ffmpeg fails to create direct3d9 device on shell window

https://bugs.winehq.org/show_bug.cgi?id=38196

Bug ID: 38196
           Summary: ffmpeg fails to create direct3d9 device on shell
                    window
           Product: Wine
           Version: 1.7.37
          Hardware: x86
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: -unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
      Distribution: ---

Created attachment 50972
  --> https://bugs.winehq.org/attachment.cgi?id=50972
wine test to reproduce the issue

If you enable DXVA2 in ffmpeg, the creation of the Direct3D9 device fails. They
use the following code:

--------
IDirect3D9_GetAdapterDisplayMode(ctx->d3d9, adapter, &d3ddm);
d3dpp.Windowed         = TRUE;
d3dpp.BackBufferWidth  = 640;
d3dpp.BackBufferHeight = 480;
d3dpp.BackBufferCount  = 0;
d3dpp.BackBufferFormat = d3ddm.Format;
d3dpp.SwapEffect       = D3DSWAPEFFECT_DISCARD;
d3dpp.Flags            = D3DPRESENTFLAG_VIDEO;

hr = IDirect3D9_CreateDevice(ctx->d3d9, adapter, D3DDEVTYPE_HAL,
GetShellWindow(), D3DCREATE_SOFTWARE_VERTEXPROCESSING | D3DCREATE_MULTITHREADED
| D3DCREATE_FPU_PRESERVE, &d3dpp, &ctx->d3d9device);
---

The creation fails because ffmpeg tries to create the device using the shell
window, but GetShellWindow() always returns NULL as handle until some program
sets the shell window using SetShellWindow(). Wine does not provide any program
which calls SetShellWindow() and it is therefore not possible to get ffmpeg +
DXVA2 working with plain Wine.

In order to test this bug you either need to apply the DXVA2 patches from
https://github.com/wine-compholio/wine-staging/tree/master/patches/dxva2-Vid...
and enable vaapi in the registry or you can use the wine test from the
attachment.

[Bug 38196] ffmpeg fails to create direct3d9 device on shell window

https://bugs.winehq.org/show_bug.cgi?id=38196

Michael Müller  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |download, testcase
                URL|                            |http://ffmpeg.zeranoe.com/b
                   |                            |uilds/
                 CC|                            |[e-mail]

[Bug 38196] ffmpeg fails to create direct3d9 device on shell window

https://bugs.winehq.org/show_bug.cgi?id=38196

Ilya Konovalov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Ilya Konovalov  ---
Created attachment 56277
  --> https://bugs.winehq.org/attachment.cgi?id=56277
possible patch

There are other applications expecting to get "explorer.exe" window when using
GetShellWindow(). An example is described here -
http://blogs.microsoft.co.il/pavely/2011/06/18/getshellwindow-vs-getdesktopw...

A simple solution would be to check if result of GetShellWindow is about to be
null and in that case call get_desktop_window and return that instead. See
attachment.

I don't have a test for that, but I've encountered an application which hit
exactly this problem. Using this patch I've managed to fix that one.

[Bug 38196] ffmpeg fails to create direct3d9 device on shell window

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=38196

joaopa  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #2 from joaopa  ---
Created attachment 70759
  --> https://bugs.winehq.org/attachment.cgi?id=70759
Compiled test for windows

Looks like the bug is fixed in WIne-6.19. I compiled the test for windows (.exe
file) and I tested with wine.

command to compile: i686-w64-mingw32-gcc test.c -ld3d9 -luser32 -lgdi32 -o
test.exe

Can an administrator close this bug as FIXED?

[Bug 38196] ffmpeg fails to create direct3d9 device on shell window

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=38196

--- Comment #3 from joaopa  ---
Once again,

can an administrator close this bug as FIXED?

You can test by yourself. I compiled the test to ease the checking (see
previous comment).

[Bug 38196] ffmpeg fails to create direct3d9 device on shell window

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=38196

[e-mail] changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #4 from [e-mail] ---
running the precompiled test.exe gives me this: received device 0078d7d8. Wine
8.6.

[Bug 38196] ffmpeg fails to create direct3d9 device on shell window

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=38196

--- Comment #5 from joaopa  ---
That means that the device was created and everything is OK (have a look at the
source code). So you confirm the bug is fixed.