[Bug 30307] New: Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

Bug #: 30307
           Summary: Memory Allocation severely limited when running wine
                    executable via condor scheduler
           Product: Wine
           Version: 1.4-rc6
          Platform: x86-64
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: blocker
          Priority: P2
         Component: -unknown
        AssignedTo: [e-mail]
        ReportedBy: [e-mail]
    Classification: Unclassified

memory allocation problem: symptom: ~300MB allocation denied on an ubuntu linux
box with 8GB running wine1.4 (also tested wine1.5).

I've created a simplified test case taking advantage of the memory test program
contributed here (http://bugs.winehq.org/attachment.cgi?id=32366). I compile it
via:

#i586-mingw32msvc-gcc -O0 test-mem.c -o test-mem.exe

thereby creating a windows executable that runs under wine. The test program
successfully allocates memory (malloc) up to 1408 megabytes, which (while
disappointing) is completely expected and about as much as I can get in 32bit
windows.

THE PROBLEM: run the same executable via the condor scheduler
(http://research.cs.wisc.edu/condor/) and the largest allocation is 448MB!! I
have extensively investigated the issue and posted on the condor-users list, to
absolutely no avail.

I need to provide my 32-bit windows program with as much memory as possible.
WHAT TO DO? any thoughts/help appreciated.

thanks, jason

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |testcase
           Severity|blocker                     |normal

--- Comment #1 from Austin English  2012-03-29 16:02:09 CDT ---
Not a blocker.

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

--- Comment #2 from [e-mail] 2012-03-29 17:20:02 CDT ---
the plot thickens....

I just tested the same job with a different scheduler (SGE - sun grid engine)
and got the EXACT same result. maximum allocation of 448 MB!!!

so it is not specific to condor. perhaps I'll try a different flavor of linux
next.

j

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

--- Comment #3 from [e-mail] 2012-03-29 20:47:37 CDT ---
i should mention: the same C code compiled for linux doesn't hit any memory
allocation limits whether run on the command line or via any scheduler.

So this issue occurs with ANY SCHEDULED WINE JOB.

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

Vitaliy Margolen  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |minor

--- Comment #4 from Vitaliy Margolen  2012-03-29 21:32:36 CDT ---
http://bugs.winehq.org/page.cgi?id=fields.html#importance

Problem seems to me external to Wine. Programs work just fine with plain Wine
and standard system configuration. That's why it's minor.

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

--- Comment #5 from [e-mail] 2012-03-29 23:20:07 CDT ---
after spending the greater part of a week investigating this issue, i can say
most certainly it's not strictly external to wine. it's clearly an issue of
wine interacting with the environment. The operating system has no limits on
memory allocation and the problem in reproducible on multiple schedulers. My
best guess is that some aspect of the environment influences the way wine loads
libraries into memory in such a way that fragments the available space into
small pieces or loads duplicate or unnecessary libraries. There is just no
evidence that linux OS is actually refusing to provide memory.

this suggests debug messages regarding failed memory allocation would be
invaluable, but I haven't found a way to produce or find them. ( debugging
produced gigs of logs)

In my experience interaction issues can be some of the hardest to pin down.
> From reading wine forums and bug dialogues it seems like this kind of issue is

more common with wine because of the nature of the beast OS+wine+app in the
context of attempting to replicate windows, not just get it right. tough task!

i'm not experienced with filing bugs. it was blocking my work so i filed it as
a blocker. In my opinion it's not minor because running on the scheduler is
essential for my application as is ram >1GB. but whatever, major, minor i don't
mind....

workaround: i think i have a way around the problem. Instead of just running
the scheduled job, I schedule a script which ssh's right back to the machine
and runs the job. This ssh session is apparently identical (in whatever respect
is impacting wine) to the interactive shell, thus i get up to ~1.5GB ram. ugly,
but easy enough and it seems to be working in early tests..

j

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

--- Comment #6 from Alexandre Julliard  2012-03-30 03:56:44 CDT ---
Most likely some library gets loaded in the middle of the address space. Check
/proc/pid/maps to see the address space layout. Requiring huge contiguous
blocks is not a good idea anyway.

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

http://bugs.winehq.org/show_bug.cgi?id=30307

--- Comment #7 from [e-mail] 2012-03-30 11:57:22 CDT ---
thanks alexandre for the reply.

that is what i was suspecting. however the key here is: how could the manner in
which the executable is invoked (in this case by a scheduler) affect what
libraries are loaded?

regarding large blocks of memory. surprised to hear that in this 'big data' day
and age. I analyze multi-GB datasets all the time, requiring they be in-memory.

i looked at the /proc/pid/maps but will try again and post.

next challenge is to provide my 32-bit program the full 2GB memory allocation
it could receive in a 64-bit windows. possible?

thanks, jason

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

https://bugs.winehq.org/show_bug.cgi?id=30307

--- Comment #8 from Austin English  ---
This is your friendly reminder that there has been no bug activity for 2 years.
Is this still an issue in current (1.7.16 or newer) wine?

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

https://bugs.winehq.org/show_bug.cgi?id=30307

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |ABANDONED

--- Comment #9 from Austin English  ---
(In reply to Austin English from comment #8)
> This is your friendly reminder that there has been no bug activity for 2
> years. Is this still an issue in current (1.7.16 or newer) wine?

Abandoned.

[Bug 30307] Memory Allocation severely limited when running wine executable via condor scheduler

https://bugs.winehq.org/show_bug.cgi?id=30307

Austin English  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #10 from Austin English  ---
Closing.