Somewhat disappointing experiences with Winelib and WTL
Hi Wine developers and users! Over the last few days, I have been trying to find out whether the combination of WTL and Winelib could be a promising inter-platform GUI solution. While I have made some progress and got some new knowledge, I believe I have now come to a dead end where I'll have to give up that attempt. In case you are interested, here's my summary report. It might be useful to others. Regards, Claus *********************************************************** The Goal: ********* WTL on Windows gives me: - compact executables: small and fast - easy-to-install executables: no DLLs needed - programs easy to maintain and deploy - mature GUI The goal is to combine WTL/ATL (no COM) with Winelib to create a Linux-application with the same look and feel and roughly the same qualities: no DLL's, shared libraries, C++ ABI hassles, etc. Just one binary linked to libc. 1. ATL/WTL ********** ATL 3.0 (from my MSVC 6 compiler) and WTL 7.5 (from sourceforge) can be made to work. Issues are: - GCC 4 is very picky with templates and would need lots of template prototypes; I have used GCC 3.3.5 - Some template prototypes are still needed - Some MSVC-specific pragmas etc. need to be ifdef'd - A few pack pragmas need to be replaced by Wine's pack headers - Everything related to _get_uuid(class) I've commented out; I need no COM stuff - Some GCC warnings need to be turned off, or alternatively code changes need to be made - Various small changes - As more parts of WTL get used (i.e. instantiated), more changes will be required Basically, ATL+WTL can reach a state where they compile with GCC 3 and the changes are still mostly cosmetic. Some of the changes might even be incorporated in the official WTL, now that it's free. I have read the mails of Boaz Harrosh' previous changes; apparently he has had to invest much more work because of COM. With the modified ATL/WTL, I was able to compile the wizard-created sample SDI application on Linux. 2. Winelib "as is" ****************** After some digging through the Web, I've been able to link the sample SDI application against Winelib. It took somewhat longer than it should have due to (a) WTL's habit of just ATLASSERT()ing things and (b) the rather elaborate work of finding those places with wine-dbg and/or ATLTRACE()s plus WINEDEBUG=+relay. Since I'm not familiar with the Wine debugger, it would have been very helpful to be able to use gdb on the resulting executable. However, gdb reported that it didn't know the file format. ==> Wishlist to Wine: - Make gdb work on Winelib programs - Document Winelib and the build process better (the documentation is outdated, though the tools are good). All in all a reasonably successful endeavour, and the application runs stable and looks fine except for the font (that probably can be configured better). BUT: ... BIG SHOCK: That's not a single ELF executable, it's a weird mixture of PE and ELF and a .so file and a wrapper program that apparently starts wine in the background ... That doesn't fulfill my goals of building a native Linux exe with no dependencies except libc. It's also quite slow when loading. 3. Winelib statically linked **************************** Apparently Wine has made a decision a few years ago: that support for native Windows apps with all the infrastructure they need (Wine server, registry, resources, multithreading, etc.) is more important than support for porting applications to native Linux apps. That decision is obviously correct, given the large number of Windows applications that are only available as binaries and require much infrastructure. I'm aware of that, but I've still decided to give my dream of a native Linux app a try. I tried the following method: (a) Get and compile Wine 0.9.7 (b) Add rules to Makedll.rules to pack the objects into .a libraries (c) Attempt to link my program's objects against those static libs (d) Make changes and corrections and repeat I first made a reduced WTL application that does not use a resource file. I don't know if this was a good decision. The following changes were necessary to Wine: - Comment out a lot of DLLMain functions - Comment out some 16-bit functions that have name collisions - Turn on debug mode (-O0 -g) - Write a startup function main() (Wine's libwinecrt0.a didn't work for me) That might be the reason for more trouble - Link to all of libs/unicode - Link to some objects in libs/wine (I didn't link to loader, don't know if that's good) This is where the heavy trouble started: - Replace the assembly stuff in critical sections (due to the debug mode) ... my app is single-threaded - Somehow set up code pages for ansi/Unix (LocaleInit() failed, see below) - Comment out some calls to the registry where the run-time environment is registered What eventually caused me to stop is: - Somehow fake Wine Server or do the equivalent stuff or just jump over (Basically Register Class as atoms) - Somehow fake a lot of initial resources (E.g. GetSystemCursor etc.) - Overall too many modifications in interacting systems - Altogether not a "robust" feeling This is where I stopped. I don't know whether I'm "so close" or miles away, but my random code editing seemed to get out of hand. 4. What is needed ***************** What would be needed to support static linking against Winelib (if that's decided to be a supportworthy goal): - Infrastructure to build static libs - separate out DllMain - separate out some 16 bit functions - Possibility to provide heap based allocation through straight malloc() - Infrastructure to fake loading resources (for those resources that come from Winelib/system) - Infrastructure for proper initialization of modules e.g. through one setup function for each DLL and one common setup function that calls the others in some natural order The overall initialization function should be configurable so that programs that need only a few DLL's (user, gdi, crtdll, kernel, ntdll) don't need to link everything - Infrastructure to allow faking registry calls for those calls initiated by Winelib internal stuff (and possibly also ATL/WTL internal stuff) - Infrastructure to handle Window class registration etc. (i.e. reduced Wine server support in the app) What I specifically don't need/want through Winelib: - networking - filesystem access - loading of external DLL's or other libraries - C++ runtime library support *********************************************************** This is, in keywords, my experience with WTL and static Winelib linking. I'll be apreciative of any responses. Please CC: my mail address. Also, if anyone is interested in hosting patches to ATL/WTL or even just looking at them, I'll gladly provide them. Regards, Claus -- Claus Fischer <claus.fischer(a)clausfischer.com> http://www.clausfischer.com/
On Mon, 2006-02-06 at 05:08 +0100, Claus Fischer wrote:
BUT: ... BIG SHOCK: That's not a single ELF executable, it's a weird mixture of PE and ELF and a .so file and a wrapper program that apparently starts wine in the background ...
That doesn't fulfill my goals of building a native Linux exe with no dependencies except libc.
Apparently under Windows you can't build a not trivial exe which depends only on libc either. -- Dmitry.
On Mon, Feb 06, 2006 at 11:17:08AM -0600, Dmitry Timoshkov wrote: : Apparently under Windows you can't build a not trivial exe which : depends only on libc either. That's right. But with WTL, it's possible to build a reasonable sophisticated application that depends only on DLLs that are already on the system. My biggest WTL application so far, a project for the Austrian Mountain Rescue Service, has roughly 100.000 lines of code (sloccount). You can find a description on http://www.clausfischer.com/eis.en.html (a bit outdated though). When the first phase was finished, the program would still run under Windows 98 SE upwards. It now depends on the following DLL's: VERSION.DLL WS2_32.DLL KERNEL32.DLL USER32.DLL GDI32.DLL COMDLG32.DLL ADVAPI32.DLL SHELL32.DLL OLE32.DLL WININET.DLL COMCTL32.DLL On any reasonable PC with internet access (it's a networked application talking to several redundant data servers), you will find those. Therefore, Installation is as easy as copying the EXE and starting. That is a requirement, since the Montain Rescue Service is a volunteer organization with members from all kinds of social background, work environment, and Laptop infrastructure. For this reason, having WTL+Winelib for Linux would have been just too nice to think of :-) Regards, Claus -- Claus Fischer <claus.fischer(a)clausfischer.com> http://www.clausfischer.com/
On Mon, 2006-02-06 at 18:34 +0100, Claus Fischer wrote:
But with WTL, it's possible to build a reasonable sophisticated application that depends only on DLLs that are already on the system.
My biggest WTL application so far, a project for the Austrian Mountain Rescue Service, has roughly 100.000 lines of code (sloccount). You can find a description on http://www.clausfischer.com/eis.en.html (a bit outdated though).
When the first phase was finished, the program would still run under Windows 98 SE upwards. It now depends on the following DLL's:
VERSION.DLL WS2_32.DLL KERNEL32.DLL USER32.DLL GDI32.DLL COMDLG32.DLL ADVAPI32.DLL SHELL32.DLL OLE32.DLL WININET.DLL COMCTL32.DLL
On any reasonable PC with internet access (it's a networked application talking to several redundant data servers), you will find those.
You seem to underestimate what the dependencies above mean for the running Windows system. You depend on the *whole* kernel + GUI subsystems, including registry and everything else. That's true for Wine as well, just in the case of Wine you see all the dependencies, not just the imports section of your exe file. -- Dmitry.
On Mon, Feb 06, 2006 at 11:41:40AM -0600, Dmitry Timoshkov wrote: : You seem to underestimate what the dependencies above mean for the : running Windows system. You depend on the *whole* kernel + GUI : subsystems, including registry and everything else. That's true : for Wine as well, just in the case of Wine you see all the dependencies, : not just the imports section of your exe file. Sure. I understand that Windows is a whole different world where many things are done differently, and to provide a similar infrastructure on a Linux system, Wine has to go great lengths. And indeed the progress that Wine has made since I've first looked at it (1996) is incredible and totally admirable. *** However, through all that progress supporting native Windows applications (which must be the main goal, of course), the "portable Windows GUI" idea has naturally fallen by the wayside somewhat, and the usefulness of Winelib as an application porting toolkit is therefore questionable. Perhaps both goals (running as native Windows apps versus porting to native Linux apps) are mutually incompatible by nature. That is in no way meant as a discouragement or some sort of critizism. It's just a summary of the lessons learned. Regards, Claus -- Claus Fischer <claus.fischer(a)clausfischer.com> http://www.clausfischer.com/
Monday, February 6, 2006, 10:53:01 AM, Claus Fischer wrote:
On Mon, Feb 06, 2006 at 11:41:40AM -0600, Dmitry Timoshkov wrote: [skip] Perhaps both goals (running as native Windows apps versus porting to native Linux apps) are mutually incompatible by nature.
Exactly for this goal Borland spent lots of time and money to create Kylix like environment. I haven't tracked since V2 so I'm not sure if they ever made C++ Builder for Linux. But that's as close as you can get to you dream... Vitaliy Margolen.
On Mon, 2006-02-06 at 20:41 -0700, Vitaliy Margolen wrote:
Exactly for this goal Borland spent lots of time and money to create Kylix like environment. I haven't tracked since V2 so I'm not sure if they ever made C++ Builder for Linux. But that's as close as you can get to you dream...
Despite the fact the applications created by it are huge and slow. Not a very nice dream (just my HO in the late evening <g>). -- Dmitry.
Claus Fischer wrote:
Since I'm not familiar with the Wine debugger, it would have been very helpful to be able to use gdb on the resulting executable. However, gdb reported that it didn't know the file format.
==> Wishlist to Wine: - Make gdb work on Winelib programs
http://www.winehq.org/site/docs/winedev-guide/dbg-others
- Document Winelib and the build process better (the documentation is outdated, though the tools are good).
All in all a reasonably successful endeavour, and the application runs stable and looks fine except for the font (that probably can be configured better).
BUT: ... BIG SHOCK: That's not a single ELF executable, it's a weird mixture of PE and ELF and a .so file and a wrapper program that apparently starts wine in the background ...
http://wiki.winehq.org/Winelib_binaries_as_ELF_executables_not_%2eexe%2eso
That doesn't fulfill my goals of building a native Linux exe with no dependencies except libc.
Maybe your goals are wrong.
It's also quite slow when loading.
Have you tried using oprofile? Is it the Wine code that is slow or the libc dynamic loader?
4. What is needed *****************
What would be needed to support static linking against Winelib (if that's decided to be a supportworthy goal):
- Infrastructure to build static libs - separate out DllMain - separate out some 16 bit functions
The isn't going to work for any application which needs COM or that needs multimedia, due to their dependence on using DLLs as function tables.
- Possibility to provide heap based allocation through straight malloc()
What does this give you?
- Infrastructure for proper initialization of modules e.g. through one setup function for each DLL and one common setup function that calls the others in some natural order The overall initialization function should be configurable so that programs that need only a few DLL's (user, gdi, crtdll, kernel, ntdll) don't need to link everything
- Infrastructure to allow faking registry calls for those calls initiated by Winelib internal stuff (and possibly also ATL/WTL internal stuff)
- Infrastructure to handle Window class registration etc. (i.e. reduced Wine server support in the app)
You really don't want to try without wineserver support. Nearly everything depends on at least one feature that the wineserver provides. If you try to make your own version of wineserver that is somehow statically linked into the application you will have to completely change how it works and maintain that version on your own. You're doing a noble thing in trying to get your Winelib application easy to deploy, but I think you're going about it the wrong way. For maintainability, take a look at using the WINEPREFIX environment variable and use dynamic libraries. That way you can deploy upgrades of your application in a small installer, whilst not needing to update the Wine libraries. If you need a bug fix in a Wine library, then you can just update the one dynamic library that has changed. If the startup speed isn't good enough for you, you can use oprofile (or some other profiling framework) to work out which functions are taking up time. We would welcome any fixes to improve the startup speed. Optionally, you could comment out functions do things that you don't need. -- Rob Shearman
On Mon, Feb 06, 2006 at 05:49:24PM +0000, Robert Shearman wrote: : > - Make gdb work on Winelib programs : http://www.winehq.org/site/docs/winedev-guide/dbg-others : http://wiki.winehq.org/Winelib_binaries_as_ELF_executables_not_%2eexe%2eso Thank you for those hints, apparently the Winelib docs that I looked into were much more out of date :-) : Maybe your goals are wrong. :-) : >It's also quite slow when loading. : : Have you tried using oprofile? Is it the Wine code that is slow or the : libc dynamic loader? With everything in memory, I get: root(a)kepler ~ # opreport | grep wine 192 0.2808 wineserver 98 0.1433 libwine_unicode.so.1 82 0.1199 libwine.so.1 root(a)kepler ~ # opreport | grep wtl 20 0.0293 linuxwtl.exe.so Wall-clock time is roughly one second (this is a minimal application). However the CPU time may be much smaller. On Windows/VMware, I get <0.1 seconds for the same program. Claus -- Claus Fischer <claus.fischer(a)clausfischer.com> http://www.clausfischer.com/
WTL on Windows gives me: - compact executables: small and fast - easy-to-install executables: no DLLs needed
What's wrong exactly with nsis installer? It gives you one .exe file, and the installation can be 100% automatic (no clicks if you want so -- double click on the .exe and in some 10 seconds it's all installed for you). See http://nsis.sourceforge.net
- mature GUI
That'd be Qt you're thinking of, even if you don't know it yet :) I can't think of anything more cross-platform, mature or easy to use. Yes, it does link statically. Compared to Qt, WTL/MFC are ugly hacks. Once you switch to Qt, I bet you won't want to use WTL ever again. Just give it a try. http://www.trolltech.com But I'm way off topic. No, I don't have much to do with either of the projects I mention, besides using them. Cheers, Kuba
On Mon, 06 Feb 2006 05:08:28 +0100, Claus Fischer wrote:
Over the last few days, I have been trying to find out whether the combination of WTL and Winelib could be a promising inter-platform GUI solution.
It's not - Win32 in general could never be described as "promising" no matter what libraries you wrap it with. Wine is best used for porting and compatibility rather than new app development. Libraries like GTK or Qt are far better
The goal is to combine WTL/ATL (no COM) with Winelib to create a Linux-application with the same look and feel and roughly the same qualities: no DLL's, shared libraries, C++ ABI hassles, etc. Just one binary linked to libc.
Your *actual* goal is described here - you want a program that is free of dependency hell. A noble goal indeed, but the right way to do this is to investigate things like autopackage, ELF statifier, and so on. And also to accept that a program with no dependencies is perhaps not so useful after all. For instance, you can probably find GTK+ 2.2 or higher on everybodies system these days. thanks -mike
participants (6)
-
Claus Fischer -
Dmitry Timoshkov -
Kuba Ober -
Mike Hearn -
Robert Shearman -
Vitaliy Margolen