Re: CUDA wrapper

[Seth Shelnutt]

Well at least it compiled, but it isn't working. We are still getting the
message that the function isn't implemented.

Initializing Nvidia gpu library
cudaMalloc CUDAStream::Allocate failed feature is not yet implemented

Now both cudamalloc and all four cuda stream's, cudaStreamCreate, Destroy,
Query and Synchronize were implemented.
I thought maybe it was because in the spec file I had the cudaStream's as
pointers (ptr) so I switched them to long but ti didn't make a difference.
Originally the argument was "stream" but I can't get any argument but ptr
and long to pass the winegcc for spec files.

http://shelnutt.twomurs.com/patches/cuda/cuda.dll.spec

Does wine need to somehow be made aware of the presence of the
cudart.dll.sofile? We tried putting it in both the system32 and the
lib folder but it
seems also that maybe WINE needs to be made aware of it?

CUDA wrapper

[Stefan Dösinger]

Wine links against cudart.dll.so from /usr/lib/ or wherever it is. You don't
have to put it in C:\windows... .

You can put a TRACE or ERR into the cudaMalloc(or whatever) function
implementation in your code to write a message to check if the functions are
properly called. I suspect they are, and that libcudart.so writes those
errors. This would then mean that the Windows and Linux cuda libraries are
different, and some features are missing in the Linux version. If that is
true, the only thing you can do is to contact Nvidia and ask them for help

From: [e-mail] [mailto:[e-mail]]
On Behalf Of Seth Shelnutt
Sent: Wednesday, July 09, 2008 7:23 PM
To: Juan Lang; [e-mail]
Subject: Re: CUDA wrapper

Well at least it compiled, but it isn't working. We are still getting the
message that the function isn't implemented.

Initializing Nvidia gpu library
cudaMalloc CUDAStream::Allocate failed feature is not yet implemented

Now both cudamalloc and all four cuda stream's, cudaStreamCreate, Destroy,
Query and Synchronize were implemented.
I thought maybe it was because in the spec file I had the cudaStream's as
pointers (ptr) so I switched them to long but ti didn't make a difference.
Originally the argument was "stream" but I can't get any argument but ptr
and long to pass the winegcc for spec files.

http://shelnutt.twomurs.com/patches/cuda/cuda.dll.spec

Does wine need to somehow be made aware of the presence of the cudart.dll.so
file? We tried putting it in both the system32 and the lib folder but it
seems also that maybe WINE needs to be made aware of it?

CUDA wrapper

[Seth Shelnutt]

OK, I've fixed a few mistakes in the .spec file and we are getting further,
but I tried debugging the output but I am not sure what it all means.

zerix01@DeepThought:~/.wine/drive_c/Program Files/Folding@home
/Folding@home-gpu$ winedbg [e-mail]
WineDbg starting on pid 0024
start_process () at /media/md0/wine/wine/dlls/kernel32/process.c:904
0x7b877d02 start_process+0xc2
[/media/md0/wine/wine/dlls/kernel32/process.c:904] in kernel32: movl
%esi,0x0(%esp)
904 ExitThread( entry( peb ) );
Wine-dbg>n
fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex
samplers and 32 total samplers
fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers +
MAX_TEXTURES(=8) > combined_samplers
fixme:win:EnumDisplayDevicesW ((null),0,0x33f40c,0x00000000), stub!
err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr
0xf7facaaf
Invalid address (0x7b877d07 start_process+0xc7) for breakpoint 0, disabling
it
Process of pid=0024 has terminated
Wine-dbg>

I believe the key line is Invalid address (0x7b877d07 start_process+0xc7)
for breakpoint 0, disabling it . But what exactly that means I am not sure,
I mean I don't know which function it is saying is missing or messed up.
Also from the documentation and from the nvidia forums it seems that both
libraries are exactly the same, and it is said that there is no difference
in writing a program for Linux vs. Windows, but I assume that is minus the
direct3d functions, which I know the folding at home program doesn't use.

On Thu, Jul 10, 2008 at 12:01 AM, Stefan Dösinger 
wrote:
> Wine links against cudart.dll.so from /usr/lib/ or wherever it is. You
> don't have to put it in C:\windows... .
>
> You can put a TRACE or ERR into the cudaMalloc(or whatever) function
> implementation in your code to write a message to check if the functions are
> properly called. I suspect they are, and that libcudart.so writes those
> errors. This would then mean that the Windows and Linux cuda libraries are
> different, and some features are missing in the Linux version. If that is
> true, the only thing you can do is to contact Nvidia and ask them for help
>
> *From:* [e-mail] [mailto:
> [e-mail]] *On Behalf Of *Seth Shelnutt
> *Sent:* Wednesday, July 09, 2008 7:23 PM
> *To:* Juan Lang; [e-mail]
> *Subject:* Re: CUDA wrapper
>
> Well at least it compiled, but it isn't working. We are still getting the
> message that the function isn't implemented.
>
> Initializing Nvidia gpu library
> cudaMalloc CUDAStream::Allocate failed feature is not yet implemented
>
> Now both cudamalloc and all four cuda stream's, cudaStreamCreate, Destroy,
> Query and Synchronize were implemented.
> I thought maybe it was because in the spec file I had the cudaStream's as
> pointers (ptr) so I switched them to long but ti didn't make a difference.
> Originally the argument was "stream" but I can't get any argument but ptr
> and long to pass the winegcc for spec files.
>
> http://shelnutt.twomurs.com/patches/cuda/cuda.dll.spec
>
> Does wine need to somehow be made aware of the presence of the
> cudart.dll.so file? We tried putting it in both the system32 and the lib
> folder but it seems also that maybe WINE needs to be made aware of it?

CUDA wrapper

[Stefan Dösinger]

I have no idea regarding that crash, but from the rest of the log it seems
that the app is initializing a d3d device; This means you'll probably have
to implement cuda<->d3d communication

From: [e-mail] [mailto:[e-mail]]
On Behalf Of Seth Shelnutt
Sent: Saturday, July 12, 2008 7:52 PM
To: [e-mail]
Subject: Re: CUDA wrapper

OK, I've fixed a few mistakes in the .spec file and we are getting further,
but I tried debugging the output but I am not sure what it all means.

zerix01@DeepThought:~/.wine/drive_c/Program
Files/Folding@home/Folding@home-gpu$ winedbg [e-mail]
WineDbg starting on pid 0024
start_process () at /media/md0/wine/wine/dlls/kernel32/process.c:904
0x7b877d02 start_process+0xc2
[/media/md0/wine/wine/dlls/kernel32/process.c:904] in kernel32: movl
%esi,0x0(%esp)
904 ExitThread( entry( peb ) );
Wine-dbg>n
fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex
samplers and 32 total samplers
fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers +
MAX_TEXTURES(=8) > combined_samplers
fixme:win:EnumDisplayDevicesW ((null),0,0x33f40c,0x00000000), stub!
err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr
0xf7facaaf
Invalid address (0x7b877d07 start_process+0xc7) for breakpoint 0, disabling
it
Process of pid=0024 has terminated
Wine-dbg>

I believe the key line is Invalid address (0x7b877d07 start_process+0xc7)
for breakpoint 0, disabling it . But what exactly that means I am not sure,
I mean I don't know which function it is saying is missing or messed up.
Also from the documentation and from the nvidia forums it seems that both
libraries are exactly the same, and it is said that there is no difference
in writing a program for Linux vs. Windows, but I assume that is minus the
direct3d functions, which I know the folding at home program doesn't use.

On Thu, Jul 10, 2008 at 12:01 AM, Stefan Dösinger 
wrote:

Wine links against cudart.dll.so from /usr/lib/ or wherever it is. You don't
have to put it in C:\windows... .

You can put a TRACE or ERR into the cudaMalloc(or whatever) function
implementation in your code to write a message to check if the functions are
properly called. I suspect they are, and that libcudart.so writes those
errors. This would then mean that the Windows and Linux cuda libraries are
different, and some features are missing in the Linux version. If that is
true, the only thing you can do is to contact Nvidia and ask them for help

From: [e-mail] [mailto:[e-mail]]
On Behalf Of Seth Shelnutt
Sent: Wednesday, July 09, 2008 7:23 PM
To: Juan Lang; [e-mail]
Subject: Re: CUDA wrapper

Well at least it compiled, but it isn't working. We are still getting the
message that the function isn't implemented.

Initializing Nvidia gpu library
cudaMalloc CUDAStream::Allocate failed feature is not yet implemented

Now both cudamalloc and all four cuda stream's, cudaStreamCreate, Destroy,
Query and Synchronize were implemented.
I thought maybe it was because in the spec file I had the cudaStream's as
pointers (ptr) so I switched them to long but ti didn't make a difference.
Originally the argument was "stream" but I can't get any argument but ptr
and long to pass the winegcc for spec files.

http://shelnutt.twomurs.com/patches/cuda/cuda.dll.spec

Does wine need to somehow be made aware of the presence of the cudart.dll.so
file? We tried putting it in both the system32 and the lib folder but it
seems also that maybe WINE needs to be made aware of it?

CUDA wrapper

[Michael Karcher]

Am Samstag, den 12.07.2008, 20:52 -0400 schrieb Seth Shelnutt:
> zerix01@DeepThought:~/.wine/drive_c/Program Files/Folding@home/Folding@home-gpu$ winedbg [e-mail]
> WineDbg starting on pid 0024
> start_process () at /media/md0/wine/wine/dlls/kernel32/process.c:904
> 0x7b877d02 start_process+0xc2 [/media/md0/wine/wine/dlls/kernel32/process.c:904] in kernel32: movl %esi,0x0(%esp)
> 904 ExitThread( entry( peb ) );
> Wine-dbg>n
> fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex samplers and 32 total samplers
> fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers + MAX_TEXTURES(=8) > combined_samplers
> fixme:win:EnumDisplayDevicesW ((null),0,0x33f40c,0x00000000), stub!
> err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr 0xf7facaaf
> Invalid address (0x7b877d07 start_process+0xc7) for breakpoint 0, disabling it
> Process of pid=0024 has terminated
> Wine-dbg>
>
> I believe the key line is Invalid address (0x7b877d07 start_process
> +0xc7) for breakpoint 0, disabling it.

No, thats boring. Your program crahsed at address 0xf7fcaaf. IIRC code
0xc0000005 is a general protection fault. After your program has
crashed, the breakpoint wine automatically sets to catch program startup
is not valid anymore (there is no program to break anymore), that causes
the message you quoted.

Whats strange is that the debugger does not catch the exception. While
the first-chance event can be disabled, you should get a last-chance
exception catch right before the program dies. See
http://www.winehq.org/site/docs/winedev-guide/dbg-config for more info
about configuring the debugger.

Have you tried a relay trace yet?

Regards,
  Michael Karcher

CUDA wrapper

[Seth Shelnutt]

We have tried to get the trace, many different ways, but to no avail. I've
read through everything on running a trace of it and I've tried it with
different files and it works fine but when we try it with the folding client
we don't get any trace. The cudart.dll.so which is placed in the
/usr/local/lib/wine folder is being recognized by wine as we are not longer
getting the not implemented error but now it is just a matter of determining
what function it isn't liking. I've double check all the functions and they
all seem to be fine minus the 4 direct3d functions and 6 functions which
contain c++ coding. The 6 functions though involve copying memory, symbol
size, and channel format. They don't involve anything that would be needed
to start the folding client, they are all things that would cause a fault
after the GPU has started calculations, or so I am lead to believe.

As always the latest code is available at
http://shelnutt.twomurs.com/patches/cuda/ and a 7z of it all at
http://shelnutt.twomurs.com/patches/cuda.7z

zerix01@DeepThought:~/.wine/drive_c/Program Files/Folding@home
/Folding@home-gpu$ WINEDEBUG=+trace winedbg [e-mail]
WineDbg starting on pid 0016
start_process () at /media/md0/wine/wine/dlls/kernel32/process.c:904
0x7b877d02 start_process+0xc2
[/media/md0/wine/wine/dlls/kernel32/process.c:904] in kernel32: movl
%esi,0x0(%esp)
Unable to open file ''
Wine-dbg>n
fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex
samplers and 32 total samplers
fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers +
MAX_TEXTURES(=8) > combined_samplers
fixme:win:EnumDisplayDevicesW ((null),0,0x33f40c,0x00000000), stub!
err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr
0xf7f80aaf
Invalid address (0x7b877d07 start_process+0xc7) for breakpoint 0, disabling
it
Process of pid=0016 has terminated
Wine-dbg>quit
zerix01@DeepThought:~/.wine/drive_c/Program Files/Folding@home
/Folding@home-gpu$ WINEDEBUG=+trace wine [e-mail]
fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex
samplers and32 total samplers
fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers +
MAX_TEXTURES(=8) > combined_samplers
fixme:win:EnumDisplayDevicesW ((null),0,0x32f40c,0x00000000), stub!
err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr
0xf7fddaaf
zerix01@DeepThought:~/.wine/drive_c/Program Files/Folding@home
/Folding@home-gpu$

Thanks,

Seth Shelnutt

On Sun, Jul 13, 2008 at 5:26 AM, Michael Karcher <
[e-mail]> wrote:
> No, thats boring. Your program crahsed at address 0xf7fcaaf. IIRC code
> 0xc0000005 is a general protection fault. After your program has
> crashed, the breakpoint wine automatically sets to catch program startup
> is not valid anymore (there is no program to break anymore), that causes
> the message you quoted.
>
> Whats strange is that the debugger does not catch the exception. While
> the first-chance event can be disabled, you should get a last-chance
> exception catch right before the program dies. See
> http://www.winehq.org/site/docs/winedev-guide/dbg-config for more info
> about configuring the debugger.
>
> Have you tried a relay trace yet?
>
> Regards,
>   Michael Karcher

CUDA wrapper

[Stefan Dösinger]

WINEDEBUG=+trace doesn't really do anything. using WINEDEBUG=trace+all (or
just +all) will enable *lots* of debug output.

However, what you want to do is to add something like this to your cuda
wrapper:

At the beginning of the file, after the includes:

WINE_DEFAULT_DEBUG_CHANNEL(cuda);

Then in each function:

void cudaSomething(int a, const char *b) {

TRACE("(%d, %s)\n", a, b);

}

Then run your app with WINEDEBUG=+cuda

That will give you some information how far the app gets in talking to your
wrapper and the native lib

From: [e-mail] [mailto:[e-mail]]
On Behalf Of Seth Shelnutt
Sent: Monday, July 14, 2008 9:31 PM
To: Michael Karcher; [e-mail]
Subject: Re: CUDA wrapper

We have tried to get the trace, many different ways, but to no avail. I've
read through everything on running a trace of it and I've tried it with
different files and it works fine but when we try it with the folding client
we don't get any trace. The cudart.dll.so which is placed in the
/usr/local/lib/wine folder is being recognized by wine as we are not longer
getting the not implemented error but now it is just a matter of determining
what function it isn't liking. I've double check all the functions and they
all seem to be fine minus the 4 direct3d functions and 6 functions which
contain c++ coding. The 6 functions though involve copying memory, symbol
size, and channel format. They don't involve anything that would be needed
to start the folding client, they are all things that would cause a fault
after the GPU has started calculations, or so I am lead to believe.

As always the latest code is available at
http://shelnutt.twomurs.com/patches/cuda/ and a 7z of it all at
http://shelnutt.twomurs.com/patches/cuda.7z

zerix01@DeepThought:~/.wine/drive_c/Program
Files/Folding@home/Folding@home-gpu$ WINEDEBUG=+trace winedbg
[e-mail]
WineDbg starting on pid 0016
start_process () at /media/md0/wine/wine/dlls/kernel32/process.c:904
0x7b877d02 start_process+0xc2
[/media/md0/wine/wine/dlls/kernel32/process.c:904] in kernel32: movl
%esi,0x0(%esp)
Unable to open file ''
Wine-dbg>n
fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex
samplers and 32 total samplers
fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers +
MAX_TEXTURES(=8) > combined_samplers
fixme:win:EnumDisplayDevicesW ((null),0,0x33f40c,0x00000000), stub!
err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr
0xf7f80aaf
Invalid address (0x7b877d07 start_process+0xc7) for breakpoint 0, disabling
it
Process of pid=0016 has terminated
Wine-dbg>quit
zerix01@DeepThought:~/.wine/drive_c/Program
Files/Folding@home/Folding@home-gpu$ WINEDEBUG=+trace wine [e-mail]
fixme:d3d:IWineD3DImpl_FillGLCaps OpenGL implementation supports 32 vertex
samplers and32 total samplers
fixme:d3d:IWineD3DImpl_FillGLCaps Expected vertex samplers +
MAX_TEXTURES(=8) > combined_samplers
fixme:win:EnumDisplayDevicesW ((null),0,0x32f40c,0x00000000), stub!
err:seh:raise_exception Unhandled exception code c0000005 flags 0 addr
0xf7fddaaf
zerix01@DeepThought:~/.wine/drive_c/Program
Files/Folding@home/Folding@home-gpu$

Thanks,

Seth Shelnutt

On Sun, Jul 13, 2008 at 5:26 AM, Michael Karcher
 wrote:

No, thats boring. Your program crahsed at address 0xf7fcaaf. IIRC code
0xc0000005 is a general protection fault. After your program has
crashed, the breakpoint wine automatically sets to catch program startup
is not valid anymore (there is no program to break anymore), that causes
the message you quoted.

Whats strange is that the debugger does not catch the exception. While
the first-chance event can be disabled, you should get a last-chance
exception catch right before the program dies. See
http://www.winehq.org/site/docs/winedev-guide/dbg-config for more info
about configuring the debugger.

Have you tried a relay trace yet?

Regards,
 Michael Karcher

CUDA wrapper

[Michael Karcher]

Am Montag, den 14.07.2008, 23:18 -0500 schrieb Stefan Dösinger:
> WINEDEBUG=+trace doesn't really do anything. using WINEDEBUG=trace+all
> (or just +all) will enable *lots* of debug output.

Right.
> However, what you want to do is to add something like this to your
> cuda wrapper:
> At the beginning of the file, after the includes:
> WINE_DEFAULT_DEBUG_CHANNEL(cuda);
> Then in each function:
>
> void cudaSomething(int a, const char *b) {
>                 TRACE("(%d, %s)\n", a, b);
> }

This would be the long-term goal, probably, but WINEDEBUG=+relay should
automatically generate thunks in memory that do this printing (but not
only for cuda, of course, except if configured appropriately). In the
short term, this should yield a hint where to start searching more
quickly.

Regards,
  Michael Karcher

CUDA wrapper

[Seth Shelnutt]

Ok, I'm pretty sure I've got a working wrapper. Still need to implement a
few functions and the direct3d calls but nearly everything is there. If
anyone would like to test out different CUDA apps with this and report any
feedback it would be much appreciate. I don't have a CUDA enabled card so I
can't test anything.

The source files are viewable here,
http://shelnutt.twomurs.com/patches/cuda/

and as a 7z file.
http://shelnutt.twomurs.com/patches/cuda.7z

Binary file is available under
http://shelnutt.twomurs.com/patches/cuda/bin/

Thanks,

Seth Shelnutt

CUDA wrapper

[Seth Shelnutt]

It seems when using this wrapper and a cuda enabled program, it causes the
program/wine to use 100% of a CPU core, while running in windows the FaH GPU
client only takes around 10-15% at most of a CPU core. Any ideas why the
sudden jump to 100% use? It makes the systems most unusable in the normal
sense, as a desktop.

CUDA wrapper

[Stefan Dösinger]

You could use oprofile to find out where the CPU time is spent - this
behavior can be caused by a lot of issues.

Does the Cuda client work now? If so, it would be cool if we could include
the wrapper in Wine, or get it into a shape to make it easilly
redistributable and installable next to Wine.

From: [e-mail] [mailto:[e-mail]]
On Behalf Of Seth Shelnutt
Sent: Saturday, July 19, 2008 7:12 AM
To: [e-mail]
Subject: Re: CUDA wrapper

It seems when using this wrapper and a cuda enabled program, it causes the
program/wine to use 100% of a CPU core, while running in windows the FaH GPU
client only takes around 10-15% at most of a CPU core. Any ideas why the
sudden jump to 100% use? It makes the systems most unusable in the normal
sense, as a desktop.