dxgi:device tests crash on X64_64

[Paul Vriens]

Hi Henri,

I'm trying to fix at least the crashes on X64_64 for this test. The 
tests are done on a Windows 7 64bit VMware guest.

The strange thing is that when I crosscompile the tests as is (64bit of 
course) the tests don't crash.

When I add the following (first thing):

BOOL is_wow64;

IsWow64Process( GetCurrentProcess(), &is_wow64 );

the tests crashes at DXGID3D10CreateDevice() in create_device().

I added that IsWow64Process() just to see if I was indeed running the 
64bit version of the test.

Any idea what's going on?

[Henri Verbeet]

2009/12/7 Paul Vriens :
> When I add the following (first thing):
>
> BOOL is_wow64;
>
> IsWow64Process( GetCurrentProcess(), &is_wow64 );
>
> the tests crashes at DXGID3D10CreateDevice() in create_device().
>
> I added that IsWow64Process() just to see if I was indeed running the 64bit
> version of the test.
>
> Any idea what's going on?

It's a bit hard to say without debug information like a backtrace and
register information, but maybe the prototype for
DXGID3D10CreateDevice() is wrong and the stack gets corrupted. An easy
thing to try would be changing the type of "arg5" from DWORD to a
pointer. I'd expect similar problems with D3D10CoreCreateDevice().

[Paul Vriens]

On 12/07/2009 12:14 PM, Henri Verbeet wrote:
> 2009/12/7 Paul Vriens:
> > When I add the following (first thing):
> >
> > BOOL is_wow64;
> >
> > IsWow64Process( GetCurrentProcess(),&is_wow64 );
> >
> > the tests crashes at DXGID3D10CreateDevice() in create_device().
> >
> > I added that IsWow64Process() just to see if I was indeed running the 64bit
> > version of the test.
> >
> > Any idea what's going on?
>
> It's a bit hard to say without debug information like a backtrace and
> register information, but maybe the prototype for
> DXGID3D10CreateDevice() is wrong and the stack gets corrupted. An easy
> thing to try would be changing the type of "arg5" from DWORD to a
> pointer. I'd expect similar problems with D3D10CoreCreateDevice().

The attached does the trick. Is that what you meant?

[Henri Verbeet]

2009/12/7 Paul Vriens :
> The attached does the trick. Is that what you meant?

Close enough, I meant "void *arg5", but it comes down to the same
thing. Note that if you're going to send a patch for this you'll need
to change the prototypes and spec in dxgi and d3d10core as well. I can
do that as well if you want though. Does D3D10CoreCreateDevice() have
the same issue? I think the "unknown0" parameter to
D3D10CoreCreateDevice() is the same thing.

[Paul Vriens]

On 12/07/2009 01:51 PM, Henri Verbeet wrote:
> 2009/12/7 Paul Vriens:
> > The attached does the trick. Is that what you meant?
>
> Close enough, I meant "void *arg5", but it comes down to the same
> thing. Note that if you're going to send a patch for this you'll need
> to change the prototypes and spec in dxgi and d3d10core as well. I can
> do that as well if you want though. Does D3D10CoreCreateDevice() have

If you don't mind creating that patch, be my guest. Do you have a real 
Windows 64-bit box for testing?

I don't see a crash for D3D10CoreCreateDevice() but if it's stack 
corruption it could be hard to trigger at will, I guess?

[Henri Verbeet]

2009/12/7 Paul Vriens :
> If you don't mind creating that patch, be my guest. Do you have a real
> Windows 64-bit box for testing?

Ok, I'll write something. I don't have any Win64 boxes at the moment,
real or virtual :-\
> I don't see a crash for D3D10CoreCreateDevice() but if it's stack corruption
> it could be hard to trigger at will, I guess?

Yeah.

[Henri Verbeet]

How does the attached patch work?

[Paul Vriens]

On 12/07/2009 02:45 PM, Henri Verbeet wrote:
> How does the attached patch work?

Both don't crash on my Win7-64 VMware box and no problems on my Win7-32 
VMware box either.