Hi,
I've ported an application which uses a library - let's call it aceutilities.so - which in turn uses ACE library (http://www.cs.wustl.edu/~schmidt/ACE.html). ACE uses libpthread.so library. So the dependency is like this: my_application -> aceutilities.so -indirectly via ace library-> libpthread.so
Wine's scheduler/pthread.c (compiled code resides in ntdll.dll.so) defines couple of symbols, which are defined (if we don't take symbol versions into consideration) also in libpthread.so. Now you probably feel what's my problem:
After loading application into memory, dynamic linker loads wine stuff first, then when loading aceutilities.so into memory, pthread symbols are defined in module ntdll.dll.so - already loaded into memory - and so it doesn't try to load libpthread.so, because pthread symbols are already resolved.
My questions are: 1) Why Winelib uses it's own implementation of pthreads (at least it looks like this is intention)? 2) Is it possible to force linker to use symbols from libpthread.so instead of symbols from ntdll.dll.so? (I don't know how to tell linker to use symbol, e.g. pthread_cond_wait, from specified library, i.e. /lib/libpthread.so.0).
Thanks for help, Juraj
My questions are:
- Why Winelib uses it's own implementation of pthreads (at least it
looks like this is intention)?
Because Wine uses his own threading model to emulate the one used in real Windows. This means that pthread needs to be 'ported' to this model for it to integrate well with Wine.
- Is it possible to force linker to use symbols from libpthread.so
instead of symbols from ntdll.dll.so? (I don't know how to tell linker to use symbol, e.g. pthread_cond_wait, from specified library, i.e. /lib/libpthread.so.0).
Well, maybe you can, but we do not guarantee at all that the result will even work. If you use Winelib to port an application and never intend to use any native Windows DLL, it could, but that is currently not supported by Wine (ie have Wine use pthread for its threading).
Lionel
After loading application into memory, dynamic linker loads wine stuff first, then when loading aceutilities.so into memory, pthread symbols are defined in module ntdll.dll.so - already loaded into memory - and so it doesn't try to load libpthread.so, because pthread symbols are already resolved.
Right. Really, you *want* aceutilities.so to use the Wine versions. In theory, it should all work fine except the threads that aceutilities creates are "wine-aware". If it doesn't, that means the Wine pthreads emulation is not perfect yet.
My questions are:
- Why Winelib uses it's own implementation of pthreads (at least it
looks like this is intention)?
Lionel answered that...
- Is it possible to force linker to use symbols from libpthread.so
instead of symbols from ntdll.dll.so? (I don't know how to tell linker to use symbol, e.g. pthread_cond_wait, from specified library, i.e. /lib/libpthread.so.0).
Unfortunately not currently. The symbol fixup semantics of ELF were designed to be easily understandable by old-skool UNIX-heads for whom dynamic linking was enough of a shock to the system as it is. That means despite the dynamic linkage loading model, symbol fixup still uses a static linking model - ie the linker always links a symbol to the first definition found, regardless of the current objects position in the link tree.
This causes all kinds of problems in Linux these days, primarily when two major version of a library are linked into the same process image and conflict.... the "official" solution is to use symbol versioning for everything, as then the symbol is always connected to the correct version at any rate, though it just invents yet another namespace for shlibs to fight over <sigh>
The real solution IMHO is to implement Solaris style group and direct fixup semantics in rtld. The glibc guys do want this, but it's not near the top of their priority list.
thanks -mike (who has a love/hate relationship with ELF :)
Thanks, Mike and Lionel.
"Mike Hearn" mike@theoretic.com wrote:
Right. Really, you *want* aceutilities.so to use the Wine versions. In theory, it should all work fine except the threads that aceutilities creates are "wine-aware". If it doesn't, that means the Wine pthreads emulation is not perfect yet.
This seems to be the case, otherwise we would have never noticed the pthread emulation. We didn't really dig into what the problem was yet though.
Is any of the pthread emulation necessary if we're only using winelib, not the native emulation? I read some things that made it sound like one of the big problems was native windows code using one of the registers that pthreads reserves. Since non-x86 platforms seem to use the native pthread library, maybe we could coax our copy of winelib into using native pthreads also, but imposing the restriction that we can't link to windows-native dlls.
Our application is just using winelib; there are no emulated dlls or exes. In addition, if it helps, only the main thread of our program calls any winelib functions. Actually, right now the helper threads to call PostMessage() to request the main thread to do some work, but I think we could replace the PostMessage() calls with a socket or some other mechanism. Our program doesn't create any windows threads via CreateThread(), So, if we promised not to call any winelib code from any thread that wine doesn't know about, would that take us out of harm's way? If that's the case, we could probably link ACE to a static version of the pthread library, and everything would just work.
If we could have both versions of the pthread library co-existing in the same process, we were also thinking that we might be able to rename the wine versions of the pthread functions to something like wine_pthread_create(), and then do "#define pthread_create wine_pthread_create", but it sounds like this would break other programs.
I've read that the MONO folks have figured out how to get threading working, but it sounds like their solution was very complicated (although I haven't tried to really understand what they did), and I'd like to avoid that if we possibly can.
Thanks,
Eric Frias efrias@syncad.com
This seems to be the case, otherwise we would have never noticed the pthread emulation. We didn't really dig into what the problem was yet though.
It might be worth doing that.
Is any of the pthread emulation necessary if we're only using winelib, not the native emulation? I read some things that made it sound like one of the big problems was native windows code using one of the registers that pthreads reserves.
Well threading is kind of hairy. My understanding of the problem (though I'm in no way a Wine guru) is that the Win32 threading model has primitives that can't be mapped onto pthreads, so rather than try and do that (and fail) Wine does its own threading and then maps pthreads to Win32/Wine threads for Linux-native code that wishes to use it. So, I don't think it makes much difference whether you use native DLLs or not - code written to Win32 may make use of threading abilities that pthreads just doesn't have.
Since non-x86 platforms seem to use the native pthread library,
Yeah, I've heard something like this as well. IIRC it was part of the work done to get Wine going on PPC..... not sure though.
maybe we could coax our copy of winelib into using native pthreads also, but imposing the restriction that we can't link to windows-native dlls.
Perhaps. I think it's more a restriction on what the APIs can do though.
In addition, if it helps, only the main thread of our program calls any winelib functions.
Yes, I think that might help. Only threads that use Win32 need to use the Wine pthread mapping as far as I'm aware (but again, could be wrong).
Our program doesn't create any windows threads via CreateThread(), So, if we promised not to call any winelib code from any thread that wine doesn't know about, would that take us out of harm's way? If that's the case, we could probably link ACE to a static version of the pthread library, and everything would just work.
Try it :) But try to find out why the Wine pthreads mapping doesn't work first.
If we could have both versions of the pthread library co-existing in the same process, we were also thinking that we might be able to rename the wine versions of the pthread functions to something like wine_pthread_create(), and then do "#define pthread_create wine_pthread_create", but it sounds like this would break other programs.
It might well do.... if WineLib (win32) code interacts with threads that Wine doesn't know about things would probably break.
I've read that the MONO folks have figured out how to get threading working, but it sounds like their solution was very complicated (although I haven't tried to really understand what they did), and I'd like to avoid that if we possibly can.
The Mono team produced an enormous patch that basically commented out anything that got in their way :) It produced a separate set of binaries. I don't really grok how they did it, but this issue has been discussed on wine-devel many times now. Basically Wine has to control elements of the process state for various reasons, which is why WineLib apps aren't just ./foo, they are `wine foo.exe.so`. The Mono patches allowed it to be dlopened().
thanks -mike
Mike Hearn wrote:
Our program doesn't create any windows threads via CreateThread(), So, if we promised not to call any winelib code from any thread that wine doesn't know about, would that take us out of harm's way? If that's the case, we could probably link ACE to a static version of the pthread library, and everything would just work.
Try it :) But try to find out why the Wine pthreads mapping doesn't work first.
I didn't dig in it a lot yet, but I think the main problem is that winelib's pthread emulation doesn't implement pthread_cond_*() calls (e.g. pthread_cond_wait) - I will know more soon. Reason why we wanted to use pthreads is that ACE is tested with native pthreads.
juraj
-
Eric Frias -
Juraj Hercek -
Lionel Ulmer -
Mike Hearn