Re: Wine Emulation: Swapping functions
"Pierre d'Herbemont" <stegefin(a)free.fr> writes:
Hi Alexandre,
This patch allows to load a PE exe to memory. You may notice that there is no "mass" byte swapping. The technics used here is a bit developer here: http://stegefin.free.fr/wine/ You might prefer a more conventional way of doing the byte swapping, in this case tell me. We might still need this approach since it could efficient for sharing data accross the LE exe and the BE winelib.
It doesn't seem very efficient to trap on every memory access... Besides, I don't see how this can possibly work, there is no guarantee that the compiler is going to generate accesses that always match the size of the requested type, memcpy() being the obvious example. I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs. -- Alexandre Julliard julliard(a)winehq.org
I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs.
Certainly dealing with the syscalls alone is a lot easier than flipping on every Windows API. You may wish to talk to TransGaming. They've actually *done* what you guys are trying to do, and run x86 software/games on a Mac. IIRC they found it unworkable (due to endianness conversion) even when using virtualization software more advanced than QEMU, however they were focussing on games where performance is critical. So maybe it doesn't matter to you guys. I think they were only converting the syscalls but I don't really remember .... Gav told me various stories of their Mac porting efforts at WineConf. The problem with running x86 Wine in QEMU is that it's harder to access the OS X APIs. I guess you could use some kind of IPC to do it but you'd end up recreating the X protocol for Quartz which seems a bit silly (amongst other things).
Mike Hearn wrote:
I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs.
Certainly dealing with the syscalls alone is a lot easier than flipping on every Windows API. You may wish to talk to TransGaming. They've actually *done* what you guys are trying to do, and run x86 software/games on a Mac. IIRC they found it unworkable (due to endianness conversion) even when using virtualization software more advanced than QEMU, however they were focussing on games where performance is critical. So maybe it doesn't matter to you guys.
I agree that flipping the syscalls is a much more promising approach than the WIN32 API. I'm also thinking that there is bound to be some relevant code around aleady (perhaps even in QEMU). While the current phase of the Darwine effort is concerned with making something that works, naturally performance is an important issue. In fact one of the architectural motivations for Darwine is that it will be able to achieve much higher performance than approaches that virtualize the OS (in addition to the objectives of not having to run Windows).
I think they were only converting the syscalls but I don't really remember .... Gav told me various stories of their Mac porting efforts at WineConf.
It would be interesting to chat with those folks indeed. I take it that given that their product is dependent on Wine, which is GPL, the relevant code which enables it to run on Mac OS X must be available as well? That would dramatically simplify what we need to do!
The problem with running x86 Wine in QEMU is that it's harder to access the OS X APIs. I guess you could use some kind of IPC to do it but you'd end up recreating the X protocol for Quartz which seems a bit silly (amongst other things).
Hmmm, not sure what you mean there. Initially we patched winegcc to make the OS X API accessible, but at Alexandre's suggestion to Pierre we've switched to binding to the symbols dynamically. Of course using the native Carbon API is necessary, otherwise we're stuck in the X11 box which is as foreign to the Mac user as having Windows running via VPC. Jim White
I agree that flipping the syscalls is a much more promising approach than the WIN32 API. I'm also thinking that there is bound to be some relevant code around aleady (perhaps even in QEMU).
Yes QEMU can do syscall conversion iirc, at least in some modes.
While the current phase of the Darwine effort is concerned with making something that works, naturally performance is an important issue. In fact one of the architectural motivations for Darwine is that it will be able to achieve much higher performance than approaches that virtualize the OS (in addition to the objectives of not having to run Windows).
I don't think you could get much higher performance than eg, Virtual PC to be honest. Maybe if you developed some way of pre-converting the binaries beforehand rather than doing it on the fly, but VPC type systems have the advantage of having very limited interaction points between the host OS and the client OS which can be tightly optimized (eg, video ram), so they can perform a lot of tricks you can't do if you want to integrate better with the OS.
I think they were only converting the syscalls but I don't really remember .... Gav told me various stories of their Mac porting efforts at WineConf.
It would be interesting to chat with those folks indeed. I take it that given that their product is dependent on Wine, which is GPL, the relevant code which enables it to run on Mac OS X must be available as well? That would dramatically simplify what we need to do!
Please do some basic research, otherwise you just come off sounding ignorant: a) Wine is not under the GPL and never was, it's under the LGPL b) TransGamings Cedega forked from Wine several years ago, back when it was under the X11 license which allowed that. So they don't have to contribute their code back. c) That fork was what motivated the switch to LGPL. d) Quite a few of TransGamings porting projects don't seem to involve Wine at any level. For instance Gav told me about a game they ported which used Java. Intererestingly, I think Gav thought the Java port was *harder* than Wine ports were due to the huge instability of Java on the Mac platform. He said at one point Apple pushed out an online update to Java which broke their game and then withdrew it, leaving no way for them to duplicate the environment that their customers were having problems with (amongst other problems)! So much for "write once, run anywhere" ...
Hmmm, not sure what you mean there. Initially we patched winegcc to make the OS X API accessible, but at Alexandre's suggestion to Pierre we've switched to binding to the symbols dynamically. Of course using the native Carbon API is necessary, otherwise we're stuck in the X11 box which is as foreign to the Mac user as having Windows running via VPC.
You realise that a quartzdrv would not change the fact that Windows apps look like Windows apps, right? All it changes is the way the bits drawn by Wine get to the screen. To be honest I think you'd achieve much better results by improving the quartzwm so X11 windows integrate better with the dock etc. If quartzwm is proprietary (don't recall) then writing a new WM that is equivalent and has all the features you need would be another way forward. While it's interesting from a technical perspective to do a quartzdrv, I'm really not convinced you would get better integration than just improving the X11 support in the OS, which would obviously also benefit those who use Linux/X11 apps on the Mac as well as Wine users. Anyway .. what I meant by that statement is that if you're running x86 Wine in QEMU you can't access the Carbon APIs directly. I think in the mode where you run foreign binaries QEMU will perform syscall conversion but obviously this is of limited use - the OS X APIs aren't accessible via syscalls. One thing you could do I guess is expose each OS X API that you wanted as a new "fake" syscall and then use a customized version of QEMU that converted it into a Mac API call, ie marshal the calls in and out of the emulated environment. You'd still pay a performance penalty but that's unavoidable. I'm not sure how you'd deal with callbacks.
Alexandre, As a conclusion, would accept that we swap (using the regular way) the headers in order to load exe on a Big Endian host? (since the exe headers are Little Endian for each NT implementation) Pierre PS : the exe header is Little Endian even on a PowerPC NT exe, since the PowerPC NT implementation was actually running on Little Endian mode, so we won't be able to run PowerPC NT exe on Mac OS X.
"Pierre d'Herbemont" <stegefin(a)free.fr> writes:
As a conclusion, would accept that we swap (using the regular way) the headers in order to load exe on a Big Endian host? (since the exe headers are Little Endian for each NT implementation)
Pierre
PS : the exe header is Little Endian even on a PowerPC NT exe, since the PowerPC NT implementation was actually running on Little Endian mode, so we won't be able to run PowerPC NT exe on Mac OS X.
So why do you want to swap the headers, if you can't do anything with the exe anyway? -- Alexandre Julliard julliard(a)winehq.org
On 25 août 04, at 18:48, Alexandre Julliard wrote:
"Pierre d'Herbemont" <stegefin(a)free.fr> writes:
As a conclusion, would accept that we swap (using the regular way) the headers in order to load exe on a Big Endian host? (since the exe headers are Little Endian for each NT implementation)
Pierre
PS : the exe header is Little Endian even on a PowerPC NT exe, since the PowerPC NT implementation was actually running on Little Endian mode, so we won't be able to run PowerPC NT exe on Mac OS X.
So why do you want to swap the headers, if you can't do anything with the exe anyway?
Because I plan to do something with it later. Pierre
N.B. The only substantive paragraph here is the last one, so folks interested in the X86/PPC linking problem should just jump down there. Mike Hearn wrote:
I agree that flipping the syscalls is a much more promising approach than the WIN32 API. I'm also thinking that there is bound to be some relevant code around aleady (perhaps even in QEMU).
Yes QEMU can do syscall conversion iirc, at least in some modes.
While the current phase of the Darwine effort is concerned with making something that works, naturally performance is an important issue. In fact one of the architectural motivations for Darwine is that it will be able to achieve much higher performance than approaches that virtualize the OS (in addition to the objectives of not having to run Windows).
I don't think you could get much higher performance than eg, Virtual PC to be honest. Maybe if you developed some way of pre-converting the binaries beforehand rather than doing it on the fly, but VPC type systems have the advantage of having very limited interaction points between the host OS and the client OS which can be tightly optimized (eg, video ram), so they can perform a lot of tricks you can't do if you want to integrate better with the OS.
Obviously I completely disagree. Such tweaking at the device layer is stuck with emulating all the code all the time plus driver stacking and will never achieve native performance. Darwine's plan for integrated emulation means that it has the potential for native performance with the only penalty being for application code which must be emulated. And because we change the loader it can be precompiled delivering performance which could be several times faster than what a VPC/QEMU OS (or user mode) virtualization can do.
I think they were only converting the syscalls but I don't really remember .... Gav told me various stories of their Mac porting efforts at WineConf.
It would be interesting to chat with those folks indeed. I take it that given that their product is dependent on Wine, which is GPL, the relevant code which enables it to run on Mac OS X must be available as well? That would dramatically simplify what we need to do!
Please do some basic research, otherwise you just come off sounding ignorant:
Of course I'm ignorant of the most everything regarding Wine history. How is that germane to anything? Your hostile attitude is why I try to limit Darwine discussion on wine-devel only to wine-specific issues.
a) Wine is not under the GPL and never was, it's under the LGPL
Oh please.
b) TransGamings Cedega forked from Wine several years ago, back when it was under the X11 license which allowed that. So they don't have to contribute their code back. c) That fork was what motivated the switch to LGPL.
Well that makes sense. I couldn't make heads or tails of their licensing. Although it does say something about the Wine of WineX 2 being LGPL. In any case it seemed clear that whatever they had is not free. Also I can't see why they would want to do any X86 emulation at all since games do usually need maximum performance. I have to figure that they are using Winelib. Of course as you say I am ignorant of the facts, although I would be delighted if any Transgaming or related folks would help with Darwine.
d) Quite a few of TransGamings porting projects don't seem to involve Wine at any level. For instance Gav told me about a game they ported which used Java.
Intererestingly, I think Gav thought the Java port was *harder* than Wine ports were due to the huge instability of Java on the Mac platform. He said at one point Apple pushed out an online update to Java which broke their game and then withdrew it, leaving no way for them to duplicate the environment that their customers were having problems with (amongst other problems)! So much for "write once, run anywhere" ...
That was indeed true before OS X. Typically ironic/tragic of Apple that the machines most in need of WORA had lousy Java. Fortunately on OS X the Java is at least as good and in some cases (such as Swing performance) is better than on Windows.
Hmmm, not sure what you mean there. Initially we patched winegcc to make the OS X API accessible, but at Alexandre's suggestion to Pierre we've switched to binding to the symbols dynamically. Of course using the native Carbon API is necessary, otherwise we're stuck in the X11 box which is as foreign to the Mac user as having Windows running via VPC.
You realise that a quartzdrv would not change the fact that Windows apps look like Windows apps, right?
This is another area that integration opens up for innovation.
All it changes is the way the bits drawn by Wine get to the screen. To be honest I think you'd achieve much better results by improving the quartzwm so X11 windows integrate better with the dock etc. If quartzwm is proprietary (don't recall) then writing a new WM that is equivalent and has all the features you need would be another way forward.
While it's interesting from a technical perspective to do a quartzdrv, I'm really not convinced you would get better integration than just improving the X11 support in the OS, which would obviously also benefit those who use Linux/X11 apps on the Mac as well as Wine users.
I agree that a really good WM for OS X X11 could make X11 applications almost behave like decent Mac applications. I'm sure that there are folks who like X11 and OS X working on it. Apple's support for X11 is a delightful surprise. Of course even so, as I said to begin with, such driver layering is devastating to performance and any hope of order-of-magnitude speed up.
Anyway .. what I meant by that statement is that if you're running x86 Wine in QEMU you can't access the Carbon APIs directly. I think in the mode where you run foreign binaries QEMU will perform syscall conversion but obviously this is of limited use - the OS X APIs aren't accessible via syscalls.
Yes, quite so.
One thing you could do I guess is expose each OS X API that you wanted as a new "fake" syscall and then use a customized version of QEMU that converted it into a Mac API call, ie marshal the calls in and out of the emulated environment. You'd still pay a performance penalty but that's unavoidable. I'm not sure how you'd deal with callbacks.
The ideal method would be to handle this with a trick loader that can link to PPC code which is escaped from the X86 emulation. We can automatically generate X86 DLLs for the PPC dynamic libraries (and hence the OS X APIs) which have little emulation escape thunks in them. There is zero performance penalty since the emulator can compile such escapes as direct native calls. Callbacks use a similar sort of little thunk which calls the emulator with a given X86 address. I would really like to get away from having to write all the little wrappers to deal with the namespace problem. I believe we can solve that with a suitable naming convention as we began to do with the winegcc patch. Alternatively we could go with with dynamic binding as proposed by Alexandre and have a syscall (or emulation escape) to call PPC code. This is probably the easiest to implement but leaves us with extra call overhead (although apparently this is something that is currently done in Wine and is quite small). By using suitable macros we can make the source independent of which method is used. The thing we don't get away from is that any code which calls OS X routines from X86 code *will* have to deal with endianess. There is overhead here, but we can be careful to use code that the emulator does a good job translating. Of course if there is substantial code (which would be unusual) it can simply be compiled as a PPC dynamic library. As for swapping functions, the preceding discussion reiterates my belief that we should confine such fussy business to the relatively small bit of code that calls into OS X, will be relatively stable over the long haul, and over which we have full control. And as for the concern over having Wine be X86 and hence incurring emulation overhead, this would be one of the first bits of code which would be a candidate for having its emulator-compiled code cached. Jim White
Jim White <jim(a)pagesmiths.com> writes:
Mike Hearn wrote:
Please do some basic research, otherwise you just come off sounding ignorant:
Of course I'm ignorant of the most everything regarding Wine history. How is that germane to anything?
Your hostile attitude is why I try to limit Darwine discussion on wine-devel only to wine-specific issues.
Please don't let that discourage you. Mike's comment was clearly uncalled for, but that's not the general attitude here, you guys are very much welcome on wine-devel. So please don't hesitate to CC wine-devel on anything that can be of interest, even if it's not strictly a Wine issue; this makes it possible for us to catch potential problems earlier on, and it helps us to get some context so that when you submit a patch we have a better idea of the big picture behind the patch. -- Alexandre Julliard julliard(a)winehq.org
Le 25 août 04, à 23:01, Jim White a écrit :
I would really like to get away from having to write all the little wrappers to deal with the namespace problem. I believe we can solve that with a suitable naming convention as we began to do with the winegcc patch.
I think we can forget this solution, loading symbol at launch time works like a charm :) (BTW it was indeed winebuild which was patched) Pierre.
Pierre d'Herbemont wrote:
Le 25 août 04, à 23:01, Jim White a écrit :
I would really like to get away from having to write all the little wrappers to deal with the namespace problem. I believe we can solve that with a suitable naming convention as we began to do with the winegcc patch.
I think we can forget this solution, loading symbol at launch time works like a charm :)
(BTW it was indeed winebuild which was patched)
Pierre.
While I think that dynamic binding can work, I'm concerned about calling conventions between X86 and PPC. We'll need to look into what those are and what QEMU uses. It may turn out that to get things to line up correctly and not have anything get clobbered, we might not be able to allow any intermediate calls. In which case we have to use an emulation escape so that the call can be translated as a direct native call rather than using a syscall. And in that case the indirection through the pointer is likely be a problem. Besides not having little wrappers for the namespace issue is not having to write syscalls for all the OS X API we want to use. I'ld much rather folks could just call the OS X API directly (albeit with carefully written code). Of course the advantage to going this (syscall) route is that it is easy to write and doesn't depend on esoteric compiler/linker/loader/emulator coordination. Of course if you're planning on compiling Wine as PPC then we're on different pages architecturally. Jim
Le 26 août 04, à 00:58, Jim White a écrit :
Of course if you're planning on compiling Wine as PPC then we're on different pages architecturally.
I still believe it is the *best* way for us to go. But I agree that as a first step we can go for qemu-darwin-i386-user, that's why I am now working a bit on qemu-darwin-i386-user. Pierre
b) TransGamings Cedega forked from Wine several years ago, back when it was under the X11 license which allowed that. So they don't have to contribute their code back. c) That fork was what motivated the switch to LGPL.
Well that makes sense. I couldn't make heads or tails of their licensing. Although it does say something about the Wine of WineX 2 being LGPL. In any case it seemed clear that whatever they had is not free. Also I can't see why they would want to do any X86 emulation at all since games do usually need maximum performance. I have to figure that they are using Winelib.
Parts of it are LGPLd, parts are X11d and parts are under the AFPL. What license things are under is pretty important you know.
That was indeed true before OS X. Typically ironic/tragic of Apple that the machines most in need of WORA had lousy Java. Fortunately on OS X the Java is at least as good and in some cases (such as Swing performance) is better than on Windows.
No, TransGaming has only ever worked with OS X. I'm afraid these stories were about quite recent behaviour (last two years or so).
This is another area that integration opens up for innovation.
No it doesn't. If you want native theming you'd have to implement a UXTHEME bridge to appearance manager. X11/Quartz is orthogonal to this issue.
Of course even so, as I said to begin with, such driver layering is devastating to performance and any hope of order-of-magnitude speed up.
The truly graphics performance critical parts for games and such go via DGA/GL anyway, so I'm not convinced about that. It's like saying X11 on Windows can't possibly beat X11 on Linux because it has to go via the GDI, yet it can sometimes happen.
The ideal method would be to handle this with a trick loader that can link to PPC code which is escaped from the X86 emulation. We can automatically generate X86 DLLs for the PPC dynamic libraries (and hence the OS X APIs) which have little emulation escape thunks in them. There is zero performance penalty since the emulator can compile such escapes as direct native calls. Callbacks use a similar sort of little thunk which calls the emulator with a given X86 address.
I would really like to get away from having to write all the little wrappers to deal with the namespace problem. I believe we can solve that with a suitable naming convention as we began to do with the winegcc patch.
In order to write such thunks you would need a type description of the entire Windows API, as pointed out by Rob. Generating this in a usable form from the headers would be incredibly hard.
Alternatively we could go with with dynamic binding as proposed by Alexandre and have a syscall (or emulation escape) to call PPC code. This is probably the easiest to implement but leaves us with extra call overhead (although apparently this is something that is currently done in Wine and is quite small).
Well more to the point it means you're emulating both the app and Wine, which won't be performant.
As for swapping functions, the preceding discussion reiterates my belief that we should confine such fussy business to the relatively small bit of code that calls into OS X, will be relatively stable over the long haul, and over which we have full control. And as for the concern over having Wine be X86 and hence incurring emulation overhead, this would be one of the first bits of code which would be a candidate for having its emulator-compiled code cached.
If you look at how many native imports Wine requires on Linux to get full functionality I don't think you can say it's relatively small. It's many thousands of functions, possibly more. It's hard to measure because so much is dynamically loaded.
Mike Hearn wrote:
...
As for swapping functions, the preceding discussion reiterates my belief that we should confine such fussy business to the relatively small bit of code that calls into OS X, will be relatively stable over the long haul, and over which we have full control. And as for the concern over having Wine be X86 and hence incurring emulation overhead, this would be one of the first bits of code which would be a candidate for having its emulator-compiled code cached.
If you look at how many native imports Wine requires on Linux to get full functionality I don't think you can say it's relatively small. It's many thousands of functions, possibly more. It's hard to measure because so much is dynamically loaded.
This is the more general project that has emerged from my initial idea for Darwine, namely X86/Linux binary compatibility for OS X. There are quite a few folks working on various combinations of Linux/BSD/Darwin/Mach compatibility, but only for the native processor. While there has been a ton of end user interest in Darwine in order to run Windows apps, I have found that integrating CPU emulation into the kernel compatibility layer attracts broad developer interest. As we work through the implementation of Darwine I believe we will have solutions for all these problems and OS X will indeed become quite the Swiss Army knife of platforms. Jim White
This is the more general project that has emerged from my initial idea for Darwine, namely X86/Linux binary compatibility for OS X.
Hmm, what is that useful for? Nearly all software for Linux is open source and can be ported or sometimes simply recompiled for any given platform. Do you have any particular programs in mind for this ability, or is it just for the cool factor? Doing a binary emulation of Linux is really, really easy compared to emulating Windows (or MacOS) because all the supporting libraries are already open source. FreeBSD does it, for instance. You only have to do the syscalls and maybe the linker/C library. IIRC there is also a "Line" program that does Linux binary emulation for Windows though for obvious reasons it was never used much ...
Mike Hearn wrote:
This is the more general project that has emerged from my initial idea for Darwine, namely X86/Linux binary compatibility for OS X.
Hmm, what is that useful for? Nearly all software for Linux is open source and can be ported or sometimes simply recompiled for any given platform. Do you have any particular programs in mind for this ability, or is it just for the cool factor?
Your knowledge of porting is evidently limited. The folks at the Fink project would be surprised to learn that what they're doing is simple. For starters there is Wine. Although X86/Darwin should work fine (as I'm guessing the loader is not needed), obviously working with your preferred platform will help with compatibility questions. Beyond that there are plenty of other applications that are dependent on ELF in various ways making porting anything but simple. When you realize that there can be more than a little gap between what users' want and can do for themselves and what developers' are interested in and actually do (and when they do it), it is very easy to understand why being able to run a supported-and-tested binary RPM can be hugely preferrable to creating yet-another-port of an application. So even though Mac OS X's popularity ensures that most useful applications will be ported and supported, the number of applications that fall into the gap is plenty big enough. Jim White
Your knowledge of porting is evidently limited. The folks at the Fink project would be surprised to learn that what they're doing is simple.
It's trivial compared to porting an app from a proprietary OS - I think the existance of Winelib spells that out. Porting an app from Linux to FreeBSD, Solaris, MacOS etc is usually a matter of making sure the software is POSIX compliance and abstracting the occasional ELFism, but not much software depends on ELF somuch as the abilities of ELF which any good OS should provide. I seem to recall some problem with Mach-O where you can have a library that is linked at startup in the usual fashion but cannot be dlopened, and the same library compiled again that can be dlopened (or the equivalent) but then not linked in normally. If that hasn't been fixed then it I'm not surprised it causes problems, it will do for Wine as well - it's a serious limitation not present in any other platform I'm aware of. But even so, these are minor issues compared to traditional porting projects. For most industrial strength Windows apps for instance "port" is usually a euphamism for "rewrite", at least when Winelib is not involved :(
When you realize that there can be more than a little gap between what users' want and can do for themselves and what developers' are interested in and actually do (and when they do it), it is very easy to understand why being able to run a supported-and-tested binary RPM can be hugely preferrable to creating yet-another-port of an application. So even though Mac OS X's popularity ensures that most useful applications will be ported and supported, the number of applications that fall into the gap is plenty big enough.
Ugh, binary RPMs are anything but supported and tested I'm afraid. They vary between distros - an RPM built for one *might* work on another but usually unless you take special steps it's hit and miss. Quite a few of the problems people encounter with Wine are due to bad or broken packaging, even on non-RPM distros like Gentoo. So don't get me started on that one :)
Jim White wrote:
Mike Hearn wrote:
I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs.
Certainly dealing with the syscalls alone is a lot easier than flipping on every Windows API. You may wish to talk to TransGaming. They've actually *done* what you guys are trying to do, and run x86 software/games on a Mac. IIRC they found it unworkable (due to endianness conversion) even when using virtualization software more advanced than QEMU, however they were focussing on games where performance is critical. So maybe it doesn't matter to you guys.
I agree that flipping the syscalls is a much more promising approach than the WIN32 API. I'm also thinking that there is bound to be some relevant code around aleady (perhaps even in QEMU).
While the current phase of the Darwine effort is concerned with making something that works, naturally performance is an important issue. In fact one of the architectural motivations for Darwine is that it will be able to achieve much higher performance than approaches that virtualize the OS (in addition to the objectives of not having to run Windows).
The only way you will be able to use the Wine libraries compiled natively while running x86 executables will be to extend the current spec format so that it works similar to IDL files. You will also need to do generate wrapper code for each COM interface too. Also, there are limitations in what you can specify (and quirks such as off-by-one errors in Windows). Perhaps you would be able to special case these and be left with a manageable task, or perhaps not. There is also the fact that the wrapper code for leaf functions may take longer to execute than the code itself. To get good performance out you will probably need to compile these types of functions (such as string manipulation functions in kernel32, shlwapi, user32) as x86. Good luck with solving these issues (although I think the estimate of getting most programs that run on Wine working in 1 year that you stated in your interview is very ambitious). Rob
Jim White wrote:
I think they were only converting the syscalls but I don't really remember .... Gav told me various stories of their Mac porting efforts at WineConf.
It would be interesting to chat with those folks indeed. I take it that given that their product is dependent on Wine, which is GPL, the relevant code which enables it to run on Mac OS X must be available as well? That would dramatically simplify what we need to do!
As Mike has pointed out, only a very small portion of our code is covered by the LGPL (and none of this is used in our Mac products), and our Mac code is not publicly available. Even if it was, it wouldn't likely help you very much with your ultimate goals. That said, you're certainly welcome to check out our Mac work in binary form. You can check out the demo of Indiana Jones and the Emperor's Tomb here: http://www.macgamefiles.com/detail.php?item=17958 I am somewhat restricted on what I can say with regards to the endianness issues with respect to cpu emulators. We're not using them in any of our Mac products that are currently shipping. There is a wide continuum of possible solutions, each of which has its strengths and weaknesses (and many of which you guys are already thinking about). Obviously, the speed of the translated code makes a key difference to which solution makes the most sense. The exact problem you're trying to solve also affects the decision: porting from source vs binaries, full application vs a small library, etc. I will point out a useful performance stat from the non-emulated gaming side of things that may give you something to consider. In a typical game, CPU time is split mostly between OpenGL and application code. With a few pathological exceptions, well under 10% of CPU time is spent in Wine code. As much as 30-40% of time can be spent in game logic and related code. If the app has a software renderer, or is more 2D oriented, that can jump to 90%+ time in application code. Clearly, any translated code would have to be very very efficient, regardless of where the line is drawn on endianness conversion. Anyhow, we're certainly watching the QEmu and Darwine project keenly, and are looking forward to seeing what you guys come up with. Take care, -Gav -- Gavriel State, Co-CEO & CTO TransGaming Technologies Inc. gav(a)transgaming.com http://www.transgaming.com Let the games begin!
On 24 août 04, at 22:50, Alexandre Julliard wrote:
"Pierre d'Herbemont" <stegefin(a)free.fr> writes:
Hi Alexandre,
This patch allows to load a PE exe to memory. You may notice that there is no "mass" byte swapping. The technics used here is a bit developer here: http://stegefin.free.fr/wine/ You might prefer a more conventional way of doing the byte swapping, in this case tell me. We might still need this approach since it could efficient for sharing data accross the LE exe and the BE winelib.
It doesn't seem very efficient to trap on every memory access... Besides, I don't see how this can possibly work, there is no guarantee that the compiler is going to generate accesses that always match the size of the requested type, memcpy() being the obvious example.
I agree. though memcpy would need a wrapper which would set the dest memory region to litlle endian, but we are coming to the limitation of this method.
I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs.
I agree it is easier, but as mike pointed we can't easily access OS X API, though I am sure there is a way. Anyway I would like to give the emulation inside method a try. I think using automatically generated wrapper can work for most APIs. For sure it won't be easy but I think there is hope in succeeding. I'll also try to add to qemu the darwin-user target, so we can see the limitation of going this way ;) Pierre.
Not sure if what I'm thinking would even work, but let's mention it. Could we add the swapping to qemu? We would end up with a x86 emulation that used big endian instead of little endian. This way, all memory accesses (from the wine api) would end up unswapped, while the method calls from the .exe files would be swapped (as they will be emulated). Is this a viable alternative? On Wed, 25 Aug 2004 13:32:22 +0200, Pierre d'Herbemont <stegefin(a)free.fr> wrote:
On 24 août 04, at 22:50, Alexandre Julliard wrote:
"Pierre d'Herbemont" <stegefin(a)free.fr> writes:
Hi Alexandre,
This patch allows to load a PE exe to memory. You may notice that there is no "mass" byte swapping. The technics used here is a bit developer here: http://stegefin.free.fr/wine/ You might prefer a more conventional way of doing the byte swapping, in this case tell me. We might still need this approach since it could efficient for sharing data accross the LE exe and the BE winelib.
It doesn't seem very efficient to trap on every memory access... Besides, I don't see how this can possibly work, there is no guarantee that the compiler is going to generate accesses that always match the size of the requested type, memcpy() being the obvious example.
I agree. though memcpy would need a wrapper which would set the dest memory region to litlle endian, but we are coming to the limitation of this method.
I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs.
I agree it is easier, but as mike pointed we can't easily access OS X API, though I am sure there is a way. Anyway I would like to give the emulation inside method a try. I think using automatically generated wrapper can work for most APIs. For sure it won't be easy but I think there is hope in succeeding. I'll also try to add to qemu the darwin-user target, so we can see the limitation of going this way ;)
Pierre.
------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ Darwine-devel mailing list Darwine-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/darwine-devel
From the limited experience I have the problem is that if a portion of code expect to have a certain type of endianness, it would make some special assumption, we won't be able to track out (See [1]). And it will be even harder to track the issues for the x86 already compiled code, in Wine we can for instance track the specific function that cause troubles and replace it with safer functions (see strcmp, strlen...). Finally I think we should only relay on call conversion, and that my swapping on-the-fly technics is not sure enough to be applied here. Pierre. [1] http://stegefin.free.fr/wine/ On 25 août 04, at 14:37, Pablo Bendersky wrote:
Not sure if what I'm thinking would even work, but let's mention it. Could we add the swapping to qemu? We would end up with a x86 emulation that used big endian instead of little endian. This way, all memory accesses (from the wine api) would end up unswapped, while the method calls from the .exe files would be swapped (as they will be emulated). Is this a viable alternative?
On Wed, 25 Aug 2004 13:32:22 +0200, Pierre d'Herbemont <stegefin(a)free.fr> wrote:
On 24 août 04, at 22:50, Alexandre Julliard wrote:
"Pierre d'Herbemont" <stegefin(a)free.fr> writes:
Hi Alexandre,
This patch allows to load a PE exe to memory. You may notice that there is no "mass" byte swapping. The technics used here is a bit developer here: http://stegefin.free.fr/wine/ You might prefer a more conventional way of doing the byte swapping, in this case tell me. We might still need this approach since it could efficient for sharing data accross the LE exe and the BE winelib.
It doesn't seem very efficient to trap on every memory access... Besides, I don't see how this can possibly work, there is no guarantee that the compiler is going to generate accesses that always match the size of the requested type, memcpy() being the obvious example.
I agree. though memcpy would need a wrapper which would set the dest memory region to litlle endian, but we are coming to the limitation of this method.
I think you are going in the wrong direction with this; the right way IMO is to compile Wine for x86 and run the whole Wine+app under the CPU emulator. Otherwise you'll have to write wrappers for each of the 15,000 APIs.
I agree it is easier, but as mike pointed we can't easily access OS X API, though I am sure there is a way. Anyway I would like to give the emulation inside method a try. I think using automatically generated wrapper can work for most APIs. For sure it won't be easy but I think there is hope in succeeding. I'll also try to add to qemu the darwin-user target, so we can see the limitation of going this way ;)
Pierre.
------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ Darwine-devel mailing list Darwine-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/darwine-devel
------------------------------------------------------- SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 _______________________________________________ Darwine-devel mailing list Darwine-devel(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/darwine-devel
participants (7)
-
Alexandre Julliard -
Gavriel State -
Jim White -
Mike Hearn -
Pablo Bendersky -
Pierre d'Herbemont -
Robert Shearman