re: Possibly crazy idea to deal with memory layout problems once and for all
it seemed that the only way to resolve it would be to get in between the application and any libraries, and the kernel.
If wine_preloader were extended to have its own implementation of all the friends of mmap(), and to have its own implementation of the dynamic linker, then in principle it could make sure only its mmap (and not the C library's) is called.
You can override mmap() in wine by just changing all the places it's called. (Having control over the source is a wonderful thing.) But if you want mmap to behave truly differently, you'd probably need to change the kernel. I seem to recall somebody was working on a linux kernel module for wine that just dealt with program loading, but I can't recall who. Perhaps he'll surface and comment on this. - Dan -- Wine for Windows ISVs: http://kegel.com/wine/isv
Dan Kegel wrote:
You can override mmap() in wine by just changing all the places it's called. (Having control over the source is a wonderful thing.) But if you want mmap to behave truly differently, you'd probably need to change the kernel.
You can do that easily after glibc has loaded, but you won't know where glibc itself was loaded into the address space. I discussed something like what Troy proposed with Vitaliy on IRC, but with the preloader looking up and overriding the mmap/munmap symbols in glibc. That would hopefully give Wine control over most mmaps and munmaps, possible save us from having to reserve memory in the preloader, and allow more control of the address space. It has a few problems though. Firstly, we'd miss mmaps done with system calls. Secondly, we'd have to make assumptions about what areas of memory the kernel would let us map, and what areas of memory were already used in the address map.
I seem to recall somebody was working on a linux kernel module for wine that just dealt with program loading, but I can't recall who. Perhaps he'll surface and comment on this.
I had a go at creating a kernel based PE loader for Linux 2.6 by forward porting parts of David Howell's Wine kernel module. It currently compiles, but that's about all. Haven't had much time to spend on it lately, because I've been busy with work ... ;) Mike
On Saturday 18 February 2006 22:47, Mike McCormack wrote:
It has a few problems though. Firstly, we'd miss mmaps done with system calls.
Yes, but how many apps actually do this?
Secondly, we'd have to make assumptions about what areas of memory the kernel would let us map, and what areas of memory were already used in the address map.
It is possible to discover this by attempting to map all areas. It would not need to be done page-by-page because we could use something akin to a binary search (actually if we really wanted to be rude about it we could grovel at /proc/self/maps, which would work (absent a debugger), but is non-portable - of course this whole technique is non-portable). If the system call fails because part of the address space it was attempting to allocate was already used, the same technique could be used to discover what was mapped and mark that space as temporarily "poisoned" (if we later run out of address space, or perhaps at the time we free neighbouring addresses, we could attempt to reclaim it).
I had a go at creating a kernel based PE loader for Linux 2.6 by forward porting parts of David Howell's Wine kernel module. It currently compiles, but that's about all.
Aren't kernel modules considered a last resort? Would it even win anything given that we still call native libraries which are then likely to call libc's mmap (directly or indirectly)?
Troy Rollo wrote:
It has a few problems though. Firstly, we'd miss mmaps done with system calls.
Yes, but how many apps actually do this?
We wouldn't care about applications, only libraries that use their own mmap system call (eg. Wine).
Secondly, we'd have to make assumptions about what areas of memory the kernel would let us map, and what areas of memory were already used in the address map.
It is possible to discover this by attempting to map all areas. It would not need to be done page-by-page because we could use something akin to a binary search (actually if we really wanted to be rude about it we could grovel at /proc/self/maps, which would work (absent a debugger), but is non-portable - of course this whole technique is non-portable).
If the system call fails because part of the address space it was attempting to allocate was already used, the same technique could be used to discover what was mapped and mark that space as temporarily "poisoned" (if we later run out of address space, or perhaps at the time we free neighbouring addresses, we could attempt to reclaim it).
Well, what's needed is a proof of concept patch so that we can explore the issues with it, show the advantages and submit it to Alexandre to finally be rejected :)
Aren't kernel modules considered a last resort? Would it even win anything given that we still call native libraries which are then likely to call libc's mmap (directly or indirectly)?
I didn't write it to solve this problem in particular... but having the kernel know about Wine's special memory layout requirements is probably the only way to solve the problem once and for all. Mike
On 2/18/06, Mike McCormack <mike(a)codeweavers.com> wrote:
Dan Kegel wrote:
You can override mmap() in wine by just changing all the places it's called. (Having control over the source is a wonderful thing.) But if you want mmap to behave truly differently, you'd probably need to change the kernel.
You can do that easily after glibc has loaded, but you won't know where glibc itself was loaded into the address space.
I discussed something like what Troy proposed with Vitaliy on IRC, but with the preloader looking up and overriding the mmap/munmap symbols in glibc. That would hopefully give Wine control over most mmaps and munmaps, possible save us from having to reserve memory in the preloader, and allow more control of the address space.
It has a few problems though. Firstly, we'd miss mmaps done with system calls. Secondly, we'd have to make assumptions about what areas of memory the kernel would let us map, and what areas of memory were already used in the address map.
I seem to recall somebody was working on a linux kernel module for wine that just dealt with program loading, but I can't recall who. Perhaps he'll surface and comment on this.
I had a go at creating a kernel based PE loader for Linux 2.6 by forward porting parts of David Howell's Wine kernel module. It currently compiles, but that's about all.
Haven't had much time to spend on it lately, because I've been busy with work ... ;)
I was thinking about the WinePluginApi SoC proposals, and how it really boils down to "make win32 and nptl threads compatible, and somehow make the regular dynamic loader handle PE files". Is there a good brain dump somewhere about what you were working on, maybe on the wine wiki somewhere? - Dan
Dan Kegel wrote:
I was thinking about the WinePluginApi SoC proposals, and how it really boils down to "make win32 and nptl threads compatible, and somehow make the regular dynamic loader handle PE files". Is there a good brain dump somewhere about what you were working on, maybe on the wine wiki somewhere?
I think win32 and NPTL threads are compatible already. The issues I see for are reserving the right memory areas, setting up the large stack, establishing a connection to the Wine server and setting up the segment registers (%fs) so that the thread and process structures are correct. If somebody were to implement this for SoC, then I'd suggest they make a small static library to link against projects that want to import Windows DLLs using Wine, and then expose LoadLibrary/GetProcAddress somehow... I don't think this is good for an SoC project, because the changes are small, and you'll spend more time arguing with Alexandre about the right way to do it than doing actual coding. Mike
On 5/10/06, Mike McCormack <mike(a)codeweavers.com> wrote:
I think win32 and NPTL threads are compatible already.
Golly.
The issues I see for are reserving the right memory areas, setting up the large stack, establishing a connection to the Wine server and setting up the segment registers (%fs) so that the thread and process structures are correct.
If somebody were to implement this for SoC, then I'd suggest they make a small static library to link against projects that want to import Windows DLLs using Wine, and then expose LoadLibrary/GetProcAddress somehow...
That all sounds pretty doable.
I don't think this is good for an SoC project, because the changes are small, and you'll spend more time arguing with Alexandre about the right way to do it than doing actual coding.
The changes may sound small to you, but they're probably a month's work for a newbie. The student could then spend the second month converting some open source project that currently has a hacked, embedded copy of Wine to use vanilla wine, maybe. I think it's perfectly appopriate, especially if we consider it a proof of concept and not something that has to be integrated into the Wine tree by end of summer to be a success. - Dan
Dan Kegel wrote:
I don't think this is good for an SoC project, because the changes are small, and you'll spend more time arguing with Alexandre about the right way to do it than doing actual coding.
The changes may sound small to you, but they're probably a month's work for a newbie. The student could then spend the second month converting some open source project that currently has a hacked, embedded copy of Wine to use vanilla wine, maybe. I think it's perfectly appopriate, especially if we consider it a proof of concept and not something that has to be integrated into the Wine tree by end of summer to be a success.
It's really Alexandre's call. It seems like he's probably going to have to spend as much time mentoring that application as he would writing the code himself. Mike
participants (3)
-
Dan Kegel -
Mike McCormack -
Troy Rollo