Hi, I been having a problem where HeapAlloc is failing even though there's plenty of virtual memory left
I've done a bit of digging but haven't been able to get to the bottom of it.
First of all I've checked that HeapAlloc will go into swap by allocating a lot of 64meg blocks and writing to them to force linux to allocate them properly, and it works fine.
Then I changed alloc.c so that CreateSubHeap writes the status returned by NtAllocateVirtualMemory and I get C00017 STATUS_NO_MEMORY. HEAP_CreateSubHeap Could not allocate 00410000 bytes reason: C00017 [STATUS_NO_MEMORY]
an I double checked this by putting a malloc in just after the failure, which failed.
So then I wrote a a small non-wine test program run as a seperate process to allocate ten times as much ram and ran it while just after the crash (while the debug box was still up) and it works fine.
Any ideas?
Top says
Mem: 514916k total, 388032k used, 126884k free, 4244k buffers
Swap: 401584k total, 251488k used, 150096k free, 62916k cached
Virt : 2279m RES : 150m Shared: 807m Mem : 30.0% wine-preloader
___________________________________________________________ ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com
Oliver Stieber wrote:
Hi, I been having a problem where HeapAlloc is failing even though there's plenty of virtual memory left
I've done a bit of digging but haven't been able to get to the bottom of it.
First of all I've checked that HeapAlloc will go into swap by allocating a lot of 64meg blocks and writing to them to force linux to allocate them properly, and it works fine.
Then I changed alloc.c so that CreateSubHeap writes the status returned by NtAllocateVirtualMemory and I get C00017 STATUS_NO_MEMORY. HEAP_CreateSubHeap Could not allocate 00410000 bytes reason: C00017 [STATUS_NO_MEMORY]
an I double checked this by putting a malloc in just after the failure, which failed.
So then I wrote a a small non-wine test program run as a seperate process to allocate ten times as much ram and ran it while just after the crash (while the debug box was still up) and it works fine.
Any ideas?
The most like answer is that there is so much fragmentation in your virtual address space that there is no block of contiguous memory free for whatever size is being passed into RtlCreateHeap. Inspecting /proc/<program PID>/maps will give you a clue as to how much fragmentation there is. Unfortunately, if this is your problem, there isn't a good solution to it. Windows is very good at keeping the virtual address space free by bunching all of the system DLLs up at just under the 2Gb barrier. The situation is worsened on Linux, due to prelink randomizing the address space, which is turned on on many modern Linux distros. However, I seem to remember there was a kernel patch written by Ingo Molnar that changes the way mmap and brk allocate virtual address space that could help a lot with this.
Rob
On Sat, 15 Jan 2005 15:58:30 -0600, Rob Shearman wrote:
The most like answer is that there is so much fragmentation in your virtual address space that there is no block of contiguous memory free for whatever size is being passed into RtlCreateHeap. Inspecting /proc/<program PID>/maps will give you a clue as to how much fragmentation there is. Unfortunately, if this is your problem, there isn't a good solution to it. Windows is very good at keeping the virtual address space free by bunching all of the system DLLs up at just under the 2Gb barrier. The situation is worsened on Linux, due to prelink randomizing the address space, which is turned on on many modern Linux distros. However, I seem to remember there was a kernel patch written by Ingo Molnar that changes the way mmap and brk allocate virtual address space that could help a lot with this.
Right, this is almost certainly a VMA layout issue. Try an
echo 1 >/proc/sys/vm/legacy_va_layout
and see if that makes any difference. Also try disabling execshield if present:
echo 0 >/proc/sys/kernel/exec-shield echo 0 >/proc/sys/kernel/exec-shield-randomize
and finally do an "export LD_USE_LOAD_BIAS=0" before running Wine (this should disable prelink for that process on recent glibcs).
Finally if you do NOT have a flex-mmap kernel, that may also allow the win32 heap to grow as much as necessary.
This does beg the question of how much RAM Pirates! expects to be able to allocate of course ....
thanks -mike
On Sun, 16 Jan 2005 12:53:31 +0000, Mike Hearn wrote:
Finally if you do NOT have a flex-mmap kernel, that may also allow the win32 heap to grow as much as necessary.
To clarify the flex-mmap patches should let us allocate a much larger process heap, so you may wish to experiment with applying them. But this patch seems to cause other problems ....
Right, this is almost certainly a VMA layout issue. Try an
echo 1 >/proc/sys/vm/legacy_va_layout
That's not there, (but my pc's a good 18 months old VIA/AMD).
and see if that makes any difference. Also try disabling execshield if present:
echo 0 >/proc/sys/kernel/exec-shield echo 0 >/proc/sys/kernel/exec-shield-randomize
Nope, no exec-shield
and finally do an "export LD_USE_LOAD_BIAS=0" before running Wine (this should disable prelink for that process on recent glibcs).
Will do, i've also tried playing with the vm settings which didn't help. 'swappiness, overcommit_memory, overcommit_ratio, max_map_count and min_free_kbytes '
Finally if you do NOT have a flex-mmap kernel, that may also allow the win32 heap to grow as much as necessary.
I've tried doing... int i; int j; char* data; for(i =0 ; i< 10 ; i++){ data = HeapAlloc(GetProcessHeap(),0,64*1024*1024);
/*write to the data for linux really allocates it*/ for(j = 0 ; j < 128; j++){ data[j*1024*512] = 1; }
} and that works ok.
This does beg the question of how much RAM Pirates! expects to be able to allocate of course ....
Well, there's plenty free, and other apps can allocate memory just fine.
I can possibly look at putting some pooling code into DirectX to help reduce fragmentation on that side (though there's not a lot of memory allocated, and it's mostly in the same sized blocks), or something in malloc so that it reports the exact failure.
thanks -mike
Oh, I also have an ATI graphics card.
___________________________________________________________ ALL-NEW Yahoo! Messenger - all new features - even more fun! http://uk.messenger.yahoo.com
Oliver Stieber wrote:
Finally if you do NOT have a flex-mmap kernel, that may also allow the win32 heap to grow as much as necessary.
I've tried doing... int i; int j; char* data; for(i =0 ; i< 10 ; i++){ data = HeapAlloc(GetProcessHeap(),0,64*1024*1024);
/*write to the data for linux really allocates it*/ for(j = 0 ; j < 128; j++){ data[j*1024*512] = 1; }
} and that works ok.
Is Pirates actually running out of heap space in HeapAlloc or is it trying to create huge heap through the HeapCreate function? If it's the first, then it would be really strange as the OS is saying that we can't allocate a 64KB block of virtual memory. Like I said before, /proc/<PID>/maps will tell us more.
Rob
Rob
-
Mike Hearn -
Oliver Stieber -
Rob Shearman