Mike Hearn wrote:
I have a problem, in that I'm trying to run an app which opens truly astonishing numbers of threads, sockets and because we're on Linux -> file descriptors.
For instance, on a basic run in the default configuration, it runs approximately 180 threads. A quick script that did an ls -1 in the /proc/$pid/fd directory of each running wine process (for some reason I get loads) shows that it opens a truly amazing 18,000 odd file descriptors.
In fact, in some circumstances it opens so many, that I get errors like this: wine client error:9a: pipe: Too many open files ... The problem is, that I don't know what limit is being used for this. In theory, the OPEN_MAX macro should tell me, but for some reason this is defined on my box as 256 (red hat 8), even though sysconf(_SC_OPEN_MAX) says "1024".
What does 'ulimit -n' say? That's the per-process limit. It's usually at 1024, but root can raise it for itself, and you can then su to the user you really want to run the program. You can also arrange for it to be higher by editing /etc/profile and logging in again.
18000 is impossible unless the processes' ulimit -n and /proc/sys/fs/file-max are both that or higher, unless I'm seriously confused.
I doubt Wine scales well to large numbers of threads or fd's. Maybe it's time to start seeing where the bottlenecks are...
- Dan
What does 'ulimit -n' say? That's the per-process limit. It's usually at 1024, but root can raise it for itself, and you can then su to the user you really want to run the program. You can also arrange for it to be higher by editing /etc/profile and logging in again.
Yes, it's 1024.
18000 is impossible unless the processes' ulimit -n and /proc/sys/fs/file-max are both that or higher, unless I'm seriously confused.
18000 was for all the wine processes added together. Normally when I run wine, i see in the process list "wine whatever.exe" and "wineserver". With this app, I see lots of "wine whatever.exe" processes, so I just took the sum of them.
I doubt Wine scales well to large numbers of threads or fd's. Maybe it's time to start seeing where the bottlenecks are...
I was discussing this with Mike McCormack last night on IRC. He gave me some ideas and sent me a couple of patches. As I'm not getting very far with this OLE deadlock, I think I'll look at these patches next.
thanks -mike
Mike Hearn wrote:
18000 is impossible unless the processes' ulimit -n and /proc/sys/fs/file-max are both that or higher, unless I'm seriously confused.
18000 was for all the wine processes added together. Normally when I run wine, i see in the process list "wine whatever.exe" and "wineserver". With this app, I see lots of "wine whatever.exe" processes, so I just took the sum of them.
Then I think you may have double-counted some fds. Each thread shows up as a process at the moment in Linux. To avoid double-counting, take only one of all the tasks with the same memory space.
I think the following line might do the job:
ps augwx | grep wine | grep -v grep | perl -e 'while (<STDIN>) { @x=split(" "); system("ls /proc/$x[1]/fd") if !$seen{$x[4]}++;}' | wc -l
but I haven't tested it thoroughly.
I doubt Wine scales well to large numbers of threads or fd's. Maybe it's time to start seeing where the bottlenecks are...
I was discussing this with Mike McCormack last night on IRC. He gave me some ideas and sent me a couple of patches. As I'm not getting very far with this OLE deadlock, I think I'll look at these patches next.
First order of business is a benchmark or two, I think. For what it's worth, I did a bit of searching, and found a few winsock benchmarky things:
http://www.etestinglabs.com/benchmarks/webbench/webbench.asp http://www.etestinglabs.com/benchmarks/netbench/home.asp http://www.sockets.com/samples.htm#WAL
It would be fun to compare webbench scores between the client app running on Wine, and the client app running on Windows... dunno if that would stress out the number of fd's in a single process, but it might.
- Dan
Dan Kegel wrote:
I doubt Wine scales well to large numbers of threads or fd's. Maybe it's time to start seeing where the bottlenecks are...
I was discussing this with Mike McCormack last night on IRC. He gave me some ideas and sent me a couple of patches. As I'm not getting very far with this OLE deadlock, I think I'll look at these patches next.
First order of business is a benchmark or two, I think.
Did you say you were running a Java app? OK, then: run, don't walk, to http://www.volano.com/benchmarks.html and see if you can get that to run under Wine. That is the java-threads-and-sockets benchmark par excellance. I haven't tried it myself under Wine, as I didn't think Java ran under Wine yet, but if you're running Jxta stuff, I guess Wine is ready for Volanomark.
It would be very, very interesting to compare Volanomark results for Windows and Wine on the same hardware. - Dan
-
Dan Kegel -
Mike Hearn