Hallo,
from the XILINX Ise suite (version 6.3 tested), the GUI (_pn.exe) and the web update (webupdate.exe) crash with recent CVS when run on a 2.6 Kernel (Suse 9.2, 2.6.11.1-20050307183958-default, but also the default 2.6 distribution kernel and a Athlon/ Pentium M). Both programs run fine with 2.4 (meeding a step back to Suse 9.0, but the same wine executable, only tested on the Athlon).
The problem was hard to pinpoint, but I suspected differences in HeapAlloc() return values to cause the problem, hinted by looking at the disassembly where the error happens.
Clearly there is a difference between 2.4 and 2.6 kernels: While mmap2() in kernel 2.4 in map_view() returns values in the 0x4000 0000 range, kernel 2.6 returns values like 0xb7fe9000, 0x58710000, 0xa07f0000 and 0x77abf000. A check in RtlAllocateHeap() for return values above 0x80000000 also triggers with both programs when run on 2.6 and not with 2.4.
Looking deeper, I found http://lwn.net/Articles/92281/ about flexible-mmap-update.patch and the wine-devel discussion from around new year about "Re: ptrace single-stepping change breaks Wine".
Setting /proc/sys/vm/legacy_va_layout, like proposed by Ingo, lets the app finally run.
Alexandre finally concluded the discussion by
Probably the easiest would be to have a way for an app to specify the mmap range it wants. So instead of having the kernel try to guess from brk and stack ulimit, both of which are meaningless for Win32 apps, we could set the range from "end of win32 exe" to 0x7ff0000. This would also avoid the need to preallocate everything above 0x80000000 that we currently do and that plays havoc with address space limits.
However nothing seems to have happened with regard to that problem until now. Could we revive that discussion?
Bye