CUDA wrapper

[Seth Shelnutt]

As a continuation from the question on making the FAH GPU2 Nvidia client
run, we have gotten past the detecting the card as we changed Wine's generic
output to look like an Nvidia card thanks to a few of you. Now comes the
hard part. Even when using the cudart.dll file it does not work. We get an
cudastream error about it not being implemented. It then repeats these 5
lines until the client is stopped.

Reading file work/wudata_07.tpr, VERSION 3.1.4 (single precision)
Reading file work/wudata_07.tpr, VERSION 3.1.4 (single precision)
Reading sasa-enabled ir 0 0
Initializing Nvidia gpu library
cudaMalloc CUDAStream::Allocate failed feature is not yet implemented

Martijn Berger said here (
http://www.winehq.org/pipermail/wine-devel/2008-July/067063.html) that all
that should need to be done is right a wrapper to translate the calls from
cudart.dll to libcuda.so.2.0. I suppose the place to start would be to
download the SDK's and see how much documentation is available on both the
Linux and Windows calls.

-Seth Shelnutt

[Seth Shelnutt]

> From reading the programing guide and reference manual it seems that most of

the function are the same as expected between Windows and Linux, in fact the
reference manual doesn't differentiate between the two.

http://developer.download.nvidia.com/compute/cuda/2.0-Beta2/docs/Programming...
http://developer.download.nvidia.com/compute/cuda/2.0-Beta2/docs/CudaRefMan_...

Is it possible to just symbolicly link the cudart.dll file to the
libcudart.so file? If they are expecting mostly the same function calls
should this not work? They are going to test this now and see.

I believe the main problem is just that cudart.dll is driving to access the
windows nvidia driver where this is none, but libcudart.so knows how to
access the linux driver.

[Michael Karcher]

Am Sonntag, den 06.07.2008, 18:23 -0400 schrieb Seth Shelnutt:
> Is it possible to just symbolicly link the cudart.dll file to the
> libcudart.so file? If they are expecting mostly the same function
> calls should this not work? They are going to test this now and see.

This is not going to work. PE style dynamic linking works quite
different from ELF style dynamic linking. Wine can not link native ELF
libraries to windows applications. The .dll.so files from wine are
special in being ELF files but containing extra information that allows
the Wine dynamic linker to link it into PE processes.
> I believe the main problem is just that cudart.dll is driving to
> access the windows nvidia driver where this is none, but libcudart.so
> knows how to access the linux driver.

This is right. You need at least a correct .spec file to make a wine
dll. You still have to implement a wrapper for each function, as Windows
usually uses the stdcall calling convention, whereas linux uses cdecl by
default. There might be some way to automate writing the wrapper
functions.

Regards,
  Michael Karcher

[Stefan Dösinger]

> This is right. You need at least a correct .spec file to make a wine
> dll. You still have to implement a wrapper for each function, as
> Windows
> usually uses the stdcall calling convention, whereas linux uses cdecl
> by
> default. There might be some way to automate writing the wrapper
> functions.

opengl32.dll has a perl script for that I think

[Seth Shelnutt]

I writing a wrapper, would it be correct to more or less follow this guide,
on winelib dll's? http://www.winehq.org/site/docs/winelib-guide/bindlls

I've never coded anything for Wine before so I want to make sure I do it
right from the beginning instead of having to go back and make drastic
changes.

I'll be looking at that opengl32 perl script because there is 102 pages of
functions that cudart.dll and libcudart.so both contain.

On Sun, Jul 6, 2008 at 6:53 PM, Michael Karcher <
[e-mail]> wrote:
> This is not going to work. PE style dynamic linking works quite
> different from ELF style dynamic linking. Wine can not link native ELF
> libraries to windows applications. The .dll.so files from wine are
> special in being ELF files but containing extra information that allows
> the Wine dynamic linker to link it into PE processes.
> This is right. You need at least a correct .spec file to make a wine
> dll. You still have to implement a wrapper for each function, as Windows
> usually uses the stdcall calling convention, whereas linux uses cdecl by
> default. There might be some way to automate writing the wrapper
> functions.
>
> Regards,
>   Michael Karcher

[Stefan Dösinger]

Yes, this guide is reasonable, except probably the building part. You *may*
want to write this DLL as part of Wine, although I am afraid that we don't
have a policy how to deal with non-Windows DLLs(since cuda is an Nvidia
thing, not from MS). Of course the 5.5 part doesn't apply then as well.

From: [e-mail] [mailto:[e-mail]]
On Behalf Of Seth Shelnutt
Sent: Sunday, July 06, 2008 6:29 PM
To: Michael Karcher
Cc: [e-mail]
Subject: Re: CUDA wrapper

I writing a wrapper, would it be correct to more or less follow this guide,
on winelib dll's? http://www.winehq.org/site/docs/winelib-guide/bindlls

I've never coded anything for Wine before so I want to make sure I do it
right from the beginning instead of having to go back and make drastic
changes.

I'll be looking at that opengl32 perl script because there is 102 pages of
functions that cudart.dll and libcudart.so both contain.

On Sun, Jul 6, 2008 at 6:53 PM, Michael Karcher
 wrote:

This is not going to work. PE style dynamic linking works quite
different from ELF style dynamic linking. Wine can not link native ELF
libraries to windows applications. The .dll.so files from wine are
special in being ELF files but containing extra information that allows
the Wine dynamic linker to link it into PE processes.

This is right. You need at least a correct .spec file to make a wine
dll. You still have to implement a wrapper for each function, as Windows
usually uses the stdcall calling convention, whereas linux uses cdecl by
default. There might be some way to automate writing the wrapper
functions.

Regards,
 Michael Karcher

[Seth Shelnutt]

Ok, please forgive my naivety but this is my first attempt and all,

I am lost and need come guidance, the guide is of some help and so is
looking at the opengl files. I am not sure exactly what I am trying to
write. What is the difference between a dll form Wine and a dll.so? I know
the dll.so end up in your linux /lib folder right? How does a dll
communicate to this dll.so? How am I to write this dll file so that when the
fah gpu app calls for a function the cudart.dll file acts as a link to the
libcudart.so.2.0? I don't want to attempt to rewrite the entire dll if I
don't have to. To me it just seems like I am not getting the picture. I know
we can't simply link the two files because they are in two different formats
ELF vs PE. Is there a good example on how this is done? I'm sure this has
been done before. Again looking to the opengl_norm.c file all it appears is
that every functions has been rewritten but with "wine_" in front of the
function name. Is that all I need to do with the cudart? That just seem
right.

I probably should even be messing with this stuff, but the beautify of open
source applications is that with a little help from the community I can add
in support for things I need (CUDA). If this was a closed source app, I
would have to just put in a request for it and maybe in a few months or so
they might think about adding support.

Thanks for your patience and help

On Sun, Jul 6, 2008 at 10:13 PM, Stefan Dösinger 
wrote:
> Yes, this guide is reasonable, except probably the building part. You
> *may* want to write this DLL as part of Wine, although I am afraid that we
> don't have a policy how to deal with non-Windows DLLs(since cuda is an
> Nvidia thing, not from MS). Of course the 5.5 part doesn't apply then as
> well.
>
> *From:* [e-mail] [mailto:
> [e-mail]] *On Behalf Of *Seth Shelnutt
> *Sent:* Sunday, July 06, 2008 6:29 PM
> *To:* Michael Karcher
> *Cc:* [e-mail]
> *Subject:* Re: CUDA wrapper
>
> I writing a wrapper, would it be correct to more or less follow this guide,
> on winelib dll's? http://www.winehq.org/site/docs/winelib-guide/bindlls
>
> I've never coded anything for Wine before so I want to make sure I do it
> right from the beginning instead of having to go back and make drastic
> changes.
>
> I'll be looking at that opengl32 perl script because there is 102 pages of
> functions that cudart.dll and libcudart.so both contain.
>
> On Sun, Jul 6, 2008 at 6:53 PM, Michael Karcher <
> [e-mail]> wrote:
>
> This is not going to work. PE style dynamic linking works quite
> different from ELF style dynamic linking. Wine can not link native ELF
> libraries to windows applications. The .dll.so files from wine are
> special in being ELF files but containing extra information that allows
> the Wine dynamic linker to link it into PE processes.
>
> This is right. You need at least a correct .spec file to make a wine
> dll. You still have to implement a wrapper for each function, as Windows
> usually uses the stdcall calling convention, whereas linux uses cdecl by
> default. There might be some way to automate writing the wrapper
> functions.
>
> Regards,
>  Michael Karcher

[Stefan Dösinger]

> Ok, please forgive my naivety but this is my first attempt and all,

No worries. This list is here for help in such cases(although I am not that
experienced in that area)
> I am lost and need come guidance, the guide is of some help and so is

looking at the
> opengl files. I am not sure exactly what I am trying to write. What is the

difference
> between a dll form Wine and a dll.so? I know the dll.so end up in your

linux /lib
> folder right? How does a dll communicate to this dll.so?

There are 3 file extensions:
.so: Those are ELF shared object files which Linux uses. Linux apps can link
to them, Windows apps can't due to reasons pointed out earlier
.dll: Those are PE files, which Windows apps can link to, but the Linux
dynamic loader can't parse
.dll.so: Those are Wine DLLs. Technically they are .so files, but with extra
information added for linking them with Windows apps.

A Windows app classically links to .dll and .exe files(.exe is just a .dll
with a main() function in it). Wine can use its own builtin .dll.so
implementations of a DLL. Those libraries are loaded by the Linux dynamic
loader, so a .dll.so can link to other Linux libraries(e.g. libcuda.so).
That way you can write functions that serve as Win32 API entrypoints for
Windows apps and call Linux APIs themselves.
> How am I to write this dll file so that when the fah gpu app calls for a

function the
> cudart.dll file acts as a link to the libcudart.so.2.0? I don't want to

attempt to
> rewrite the entire dll if I don't have to. To me it just seems like I am

not getting
> the picture. I know we can't simply link the two files because they are in

two
> different formats ELF vs PE. Is there a good example on how this is done?

I'm sure
> this has been done before. Again looking to the opengl_norm.c file all it

appears is
> that every functions has been rewritten but with "wine_" in front of the

function
> name. Is that all I need to do with the cudart? That just seem right.

You don't want to rewrite CUDA in the way we rewrite e.g. d3d9.dll, since
that would mean to talk to the GPU or driver directly without Nvidia's
libcudart.so.2.0. You just want to write functions that simply call the
function exported from the Linux library. For example, like in the opengl
stuff:

FancyCudaReturnValue wine_CudaSomeFunc(int a, int b) {
    return CudaSumeFunc(a, b);
}

That's not really called a "rewrite"(e.g. look at ole32 to see what would be
considered a "rewrite"), but rather a thunk. Unfortunately it can still be a
a whole lot of typing work, unless you manage to write a script to generate
the thunks.

You have to take if there are some differences between the Linux and Windows
functions, e.g.:
Win32: CudaSomething(HWND window);
Linux: CudaSomething(XWindow window);

In this case you can't just pass the Windows specific data structure
along(Don't ask me how you would solve this exact issue here since only
winex11.drv is supposed to know about the HWND<->XWindow mappings.)

You also have to take care with filenames:
Win32: CudaSomething2(const char *filename);
Linux: CudaSomething2(const char *filename);
The Linux function would not be happy if you pass it a
"C:\Path\to\some\odd\place.txt", and likewise the Windows app will not be
happy if you pass it a "/home/user/my/fancy/location.txt" file returned from
cuda. For cases like these, Wine provides functions to translate paths.

[Seth Shelnutt]

I've attached my spec file and my .c file along with the two header files
from the nvidia toolkit. I am pretty sure in the .c file I was not suppose
to use the "WINAPI" but instead I need "FancyCudaReturnValue" but I don't
know what that is. Needless to say after using "winemaker cuda" (I have it
in a folder called cuda), it doesn't compile and I get a million and one
errors. I'm sure these errors are more or less caused because I didn't quite
get it right. If you could just look at it and give me some pointers on
where and how I went wrong, it would be appreciated. You've been a big help
so far. Thank you for the explanations in the last email it got me this far
so far.

-Seth Shelnutt

[Stefan Dösinger]

Actually you want something like

retval WINAPI wine_cudaSomething(int a, int etc)

so instead of the void use the return value the function is supposed to
return

WINAPI tells the compiler about the calling convention(ie, first parameter
on the stack, in ecx, or elsewhere, who takes care about cleaning up the
stack, etc). You'll have to check the calling convention Win32 cuda uses,
but most likely WINAPI is correct. You don't have to care about the Linux
cuda calling convention, since the compiler knows about that from the Linux
cuda headers

I am also not quite sure about some constructs, like

"wine_cudaBindTexture( size_t* offset, const struct texture < T, dim,
readMode >& texRef, const void* devPtr, const struct cudaChannelFormatDesc&
desc, size_t size = UINT_MAX )" As far as I know this contains C++ or
Microsoft syntax, which is not valid in pure C.

From: Seth Shelnutt [mailto:[e-mail]] 
Sent: Monday, July 07, 2008 11:01 AM
To: Stefan Dösinger
Cc: [e-mail]
Subject: Re: CUDA wrapper

I've attached my spec file and my .c file along with the two header files
from the nvidia toolkit. I am pretty sure in the .c file I was not suppose
to use the "WINAPI" but instead I need "FancyCudaReturnValue" but I don't
know what that is. Needless to say after using "winemaker cuda" (I have it
in a folder called cuda), it doesn't compile and I get a million and one
errors. I'm sure these errors are more or less caused because I didn't quite
get it right. If you could just look at it and give me some pointers on
where and how I went wrong, it would be appreciated. You've been a big help
so far. Thank you for the explanations in the last email it got me this far
so far.

-Seth Shelnutt

[Juan Lang]

> I am also not quite sure about some constructs, like
>
> "wine_cudaBindTexture( size_t* offset, const struct texture < T, dim,
> readMode >& texRef, const void* devPtr, const struct cudaChannelFormatDesc&
> desc, size_t size = UINT_MAX )" As far as I know this contains C++ or
> Microsoft syntax, which is not valid in pure C.

Correct, this is C++.  If you're wondering about the const reference,
that's only valid in C++.  If there is such a declaration as part of
the CUDA API, then it's only accessible from C++ callers, and your
implementation must be in C++.

If you're wondering about the default value for size, that's also C++.
 Default values are only valid in a function declaration, not in a
function definition.  Therefore I think you can just omit them from
your own implementation.
--Juan