Dear Wine developers,
I am working on a little Python module which is supposed to make use of Wine. The module has a dedicated class for managing its own wineserver-session with its own prefix. It allows me to control wineserver independently of the lifetime of any Windows executable I run on top of it and to quit it when my work is done.
Starting and stopping wineserver is no problem. However, what I can not figure out is how to determine when my wineserver is actually completely loaded, up and running. After all, the loading process requires quite a bit of time. Mainly due to my intention of preventing any (hard to debug) crashes and delays when running Windows executables, I want wineserver to be fully up & loaded before I run anything on top of it. The following code snipped illustrates the problem:
os.environ['WINEPREFIX'] = '/some/path' proc_wineserver = subprocess.Popen( ['wineserver', '-f', '-p'], # run persistent in foreground stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell = False ) time.sleep(1) # Placeholder *HACK*. BETTER SOLUTION? proc_somewinexe = subprocess.Popen(['wine some.exe'], shell = False)
As a temporary hack, for which I am seeking a replacement, I simply sleep for one second. This is odd on faster PCs and really not enough on somewhat older hardware (or with slow harddrives instead of SSDs) ...
Is there a signal I can catch or a file I can check (and wait for in a loop to appear) or anything else which would allow me a clean(er) solution? Or, alternatively, where in the Wine source code do I have to look for how the "wine" command does it (?) - looking for wineserver and firing one up if there is not one before any exe file is executed?
I recently posted a question in the Wine forum with similar content: https://forum.winehq.org/viewtopic.php?f=2&t=28967
Thanks for any advise, Sebastian
On 12.06.2017 20:22, Sebastian M. Ernst wrote:
Dear Wine developers,
I am working on a little Python module which is supposed to make use of Wine. The module has a dedicated class for managing its own wineserver-session with its own prefix. It allows me to control wineserver independently of the lifetime of any Windows executable I run on top of it and to quit it when my work is done.
Starting and stopping wineserver is no problem. However, what I can not figure out is how to determine when my wineserver is actually completely loaded, up and running. After all, the loading process requires quite a bit of time. Mainly due to my intention of preventing any (hard to debug) crashes and delays when running Windows executables, I want wineserver to be fully up & loaded before I run anything on top of it. The following code snipped illustrates the problem:
os.environ['WINEPREFIX'] = '/some/path' proc_wineserver = subprocess.Popen( ['wineserver', '-f', '-p'], # run persistent in foreground stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell = False ) time.sleep(1) # Placeholder *HACK*. BETTER SOLUTION? proc_somewinexe = subprocess.Popen(['wine some.exe'], shell = False)
As a temporary hack, for which I am seeking a replacement, I simply sleep for one second. This is odd on faster PCs and really not enough on somewhat older hardware (or with slow harddrives instead of SSDs) ...
Is there a signal I can catch or a file I can check (and wait for in a loop to appear) or anything else which would allow me a clean(er) solution? Or, alternatively, where in the Wine source code do I have to look for how the "wine" command does it (?) - looking for wineserver and firing one up if there is not one before any exe file is executed?
I recently posted a question in the Wine forum with similar content: https://forum.winehq.org/viewtopic.php?f=2&t=28967
Thanks for any advise, Sebastian
I would suggest to implement the same logic as used by Wine itself. Basically, you would write a loop which periodically tries to connect to the wineserver UNIX socket, until it finally succeeds. The socket will appear at the following location (pseudocode):
info = os.stat(WINEPREFIX) socket_path = os.path.join("/tmp", ".wine-%d" % os.getuid(), "server-%x-%x" % (info.st_dev, info.st_ino))
Best regards, Sebastian
Hi Sebastian,
thanks a lot for the idea.
I tried to convert your idea into a quick & dirty piece of actual code for testing it - see below this email. Simplified into pseudo-code I do the following: - create socket object, type unix / stream (w/o reference to file/path) - try: connect to socket file - if failed, wait for x seconds and repeat last step (until timeout) - if succeeded, close the connection and let the code go ahead
It will result in a log like the following (assuming that most involved files are cached in RAM, otherwise the numbers in the "appeared" message will be somewhat higher):
[wine session] Launching wineserver ... [wine session] ... started with PID 28549 ... [wine session] ... expecting socket at /tmp/.wine-1000/server-803-690008/socket ... [wine session] ... appeared (after 0.00 seconds & 1 attempts)!
So far so good. This code works *sometimes* (rarely, however). It will always tell me that the desired socket appeared, which it actually does, but usually, most of the time, any subsequent command like "wine some.exe" or "winepath -w /path" following the launch of my wineserver will crash with this error: "wine client error:0: recvmsg: Connection reset by peer"
It seems that my attempt to connect to the socket (and disconnect from it if the connection succeeds) renders it unusable for any wine process trying to connect to it afterward. Just leaving my connection open does not change the result. What I am missing or doing wrong? Am I using the right type of socket (stream)? Is there some sort of a simple "ping" message/signal I can send to the wineserver through the socket which should result in a deterministic answer I can use for validating the connection?
Regards, Sebastian
# Status log self.log.out('[wine session] Launching wineserver ...')
# Start wine server into prepared environment self.proc_wineserver = subprocess.Popen( ['wineserver', '-f', '-p'], # run persistent in foreground stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell = False )
# Status log self.log.out('[wine session] ... started with PID %d ...' % self.proc_wineserver.pid)
# Get info on WINEPREFIX folder info_wineprefix = os.stat(self.dir_wineprefix)
# Get path of wineserver socket file socket_path = os.path.join( tempfile.gettempdir(), '.wine-%d' % os.getuid(), 'server-%x-%x' % (info_wineprefix.st_dev, info_wineprefix.st_ino), 'socket' )
# Status log self.log.out('[wine session] ... expecting socket at %s ...' % socket_path)
# Create socket client wineserver_client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) # Set a timeout wineserver_client.settimeout(1.0) # seconds # Status variable got_connection = False # Time-step wait_for_seconds = 0.01 # Timeout timeout_after_seconds = 30.0 # Already waited for ... started_waiting_at = time.time() # Connection trys tried_this_many_times = 0
# Run loop until socket appears while True:
# Does socket file exist? if os.path.exists(socket_path):
# Count attempts tried_this_many_times += 1
# Can I connect to it? try: wineserver_client.connect(socket_path) got_connection = True break except: pass
# Break to loop after timeout if time.time() >= (started_waiting_at + timeout_after_seconds): break
# Wait before trying again time.sleep(wait_for_seconds)
# Evaluate the result if not got_connection:
self.log.out( '[wine session] ... did not appear (after %0.2f seconds & %d attempts)! Quit.' % (timeout_after_seconds, tried_this_many_times) ) sys.exit()
else:
# If it worked, disconnect wineserver_client.close()
# Log status self.log.out( '[wine session] ... appeared (after %0.2f seconds & %d attempts)!' % (time.time() - started_waiting_at, tried_this_many_times) )
Am 12.06.2017 um 20:43 schrieb Sebastian Lackner:
I would suggest to implement the same logic as used by Wine itself. Basically, you would write a loop which periodically tries to connect to the wineserver UNIX socket, until it finally succeeds. The socket will appear at the following location (pseudocode):
info = os.stat(WINEPREFIX) socket_path = os.path.join("/tmp", ".wine-%d" % os.getuid(), "server-%x-%x" % (info.st_dev, info.st_ino))
Best regards, Sebastian
On Fri, Jun 16, 2017 at 6:06 PM, Sebastian M. Ernst ernst@pleiszenburg.de wrote:
Hi Sebastian,
thanks a lot for the idea.
I tried to convert your idea into a quick & dirty piece of actual code for testing it - see below this email. Simplified into pseudo-code I do the following:
- create socket object, type unix / stream (w/o reference to file/path)
- try: connect to socket file
- if failed, wait for x seconds and repeat last step (until timeout)
- if succeeded, close the connection and let the code go ahead
Not sure if I got the problem you are describing exactly but what about doing the opposite direction? Make a very simple socket program for Windows that is launched and this program connects to your python application instead. When python notices the connection it is sure that wineserver is running because it just received the connection from an app running inside Wine. Also as long as this small utility is up wineserver won't close too.
Best wishes, Bruno
Hi Sebastian,
If you disconnect before the real command is started the wineserver will assume it is no longer needed and terminate automatically. I would say this is a bug, because the user definitely does not expect that when passing "-p". Nevertheless, when you keep the connection open, it should actually work (and also does work for me, according to a quick test).
A slightly better solution which does not trigger this problem would be to wait for the wineserver lock (which will also appear in the server dir). To check a lock from Python you can use the following code snippet:
--- snip --- import fcntl import sys import struct
def getLockPID(file): """Return process ID responsible for locking a file descriptor."""
l_type = fcntl.F_WRLCK l_whence = 0 # SEEK_SET l_start = 0 l_len = 0 l_pid = 0
if sys.platform == 'linux': lock = struct.pack('hhlli', l_type, l_whence, l_start, l_len, l_pid) lock = fcntl.fcntl(file, fcntl.F_GETLK, lock) l_type, l_whence, l_start, l_len, l_pid = struct.unpack('hhlli', lock)
else: raise NotImplementedError("Platform %s not implemented" % sys.platform)
return l_pid if l_type != fcntl.F_UNLCK else None --- snip ---
Usage is like:
with open(lock_file, "rb") as fp: pid = getLockPID(fp)
There is still a small race-condition (the lock is acquired but the socket has not been created yet), but Wine will be able to deal with that. It will retry a couple of times until it finally succeeds.
BTW: As I saw, you replaced the "/tmp" with tempfile.gettempdir(), but this is actually not correct. Wineserver sockets will always be created in /tmp (hardcoded in the source) to ensure the correct socket is found, no matter which environment variables are set.
Best regards, Sebastian
On 16.06.2017 23:06, Sebastian M. Ernst wrote:
Hi Sebastian,
thanks a lot for the idea.
I tried to convert your idea into a quick & dirty piece of actual code for testing it - see below this email. Simplified into pseudo-code I do the following:
- create socket object, type unix / stream (w/o reference to file/path)
- try: connect to socket file
- if failed, wait for x seconds and repeat last step (until timeout)
- if succeeded, close the connection and let the code go ahead
It will result in a log like the following (assuming that most involved files are cached in RAM, otherwise the numbers in the "appeared" message will be somewhat higher):
[wine session] Launching wineserver ... [wine session] ... started with PID 28549 ... [wine session] ... expecting socket at /tmp/.wine-1000/server-803-690008/socket ... [wine session] ... appeared (after 0.00 seconds & 1 attempts)!
So far so good. This code works *sometimes* (rarely, however). It will always tell me that the desired socket appeared, which it actually does, but usually, most of the time, any subsequent command like "wine some.exe" or "winepath -w /path" following the launch of my wineserver will crash with this error: "wine client error:0: recvmsg: Connection reset by peer"
It seems that my attempt to connect to the socket (and disconnect from it if the connection succeeds) renders it unusable for any wine process trying to connect to it afterward. Just leaving my connection open does not change the result. What I am missing or doing wrong? Am I using the right type of socket (stream)? Is there some sort of a simple "ping" message/signal I can send to the wineserver through the socket which should result in a deterministic answer I can use for validating the connection?
Regards, Sebastian
# Status log self.log.out('[wine session] Launching wineserver ...')
# Start wine server into prepared environment self.proc_wineserver = subprocess.Popen( ['wineserver', '-f', '-p'], # run persistent in foreground stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE, shell = False )
# Status log self.log.out('[wine session] ... started with PID %d ...' % self.proc_wineserver.pid)
# Get info on WINEPREFIX folder info_wineprefix = os.stat(self.dir_wineprefix)
# Get path of wineserver socket file socket_path = os.path.join( tempfile.gettempdir(), '.wine-%d' % os.getuid(), 'server-%x-%x' % (info_wineprefix.st_dev, info_wineprefix.st_ino), 'socket' )
# Status log self.log.out('[wine session] ... expecting socket at %s ...' % socket_path)
# Create socket client wineserver_client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) # Set a timeout wineserver_client.settimeout(1.0) # seconds # Status variable got_connection = False # Time-step wait_for_seconds = 0.01 # Timeout timeout_after_seconds = 30.0 # Already waited for ... started_waiting_at = time.time() # Connection trys tried_this_many_times = 0
# Run loop until socket appears while True:
# Does socket file exist? if os.path.exists(socket_path):
# Count attempts tried_this_many_times += 1 # Can I connect to it? try: wineserver_client.connect(socket_path) got_connection = True break except: pass# Break to loop after timeout if time.time() >= (started_waiting_at + timeout_after_seconds): break
# Wait before trying again time.sleep(wait_for_seconds)
# Evaluate the result if not got_connection:
self.log.out( '[wine session] ... did not appear (after %0.2f seconds & %d attempts)! Quit.' % (timeout_after_seconds, tried_this_many_times) ) sys.exit()
else:
# If it worked, disconnect wineserver_client.close()
# Log status self.log.out( '[wine session] ... appeared (after %0.2f seconds & %d attempts)!' % (time.time() - started_waiting_at, tried_this_many_times) )
Am 12.06.2017 um 20:43 schrieb Sebastian Lackner:
I would suggest to implement the same logic as used by Wine itself. Basically, you would write a loop which periodically tries to connect to the wineserver UNIX socket, until it finally succeeds. The socket will appear at the following location (pseudocode):
info = os.stat(WINEPREFIX) socket_path = os.path.join("/tmp", ".wine-%d" % os.getuid(), "server-%x-%x" % (info.st_dev, info.st_ino))
Best regards, Sebastian
@Bruno
Not sure if I got the problem you are describing exactly but what about doing the opposite direction? Make a very simple socket program for Windows that is launched and this program connects to your python application instead. When python notices the connection it is sure that wineserver is running because it just received the connection from an app running inside Wine. Also as long as this small utility is up wineserver won't close too.
Thanks, yes, I have considered this approach (and will likely follow it if everything else does not work). I'd prefer a "pythonic" solution without adding dependencies like a cross-compiler (for compiling the utility) to my module. I am already doing this for tests, but I'd really like to keep the module itself "clean" and easy to install. Just as a theoretical question, could such a utility be implemented as a batch file which wine (cmd) could handle "out of the box"?
@Sebastian
thanks again, this is really helpful.
If you disconnect before the real command is started the wineserver will assume it is no longer needed and terminate automatically. I would say this is a bug, because the user definitely does not expect that when passing "-p".
Is this worth opening a ticket?
Nevertheless, when you keep the connection open, it should actually work (and also does work for me, according to a quick test).
Once my wineserver is up, I try to do a number of path conversion using "winepath -w /some/path" etc. first before I fire up any exe-files. Am I correct in assuming that winepath should also use my wineserver (if run with the same prefix)? Can you check whether your quick test also works with winepath (eliminating a potential source of error)? I managed to turn winepath directly into a zombie a number of times by running winepath (too early) while wineserver was still starting. It does not produce any error messages whatsoever.
A slightly better solution which does not trigger this problem would be to wait for the wineserver lock (which will also appear in the server dir).
I used your code and implemented a routine which waits for the lock-file to be locked - see below this email. Usually, it requires two attempts / about 0.01 seconds until it succeeds. The PID returned by your code matches the PID of my wineserver. The problem is, just after I have confirmed the lock, winepath becomes a zombie again and again and ... occasionally, it works. Is this a bug or a lack of understanding on my part?
There is still a small race-condition (the lock is acquired but the socket has not been created yet), but Wine will be able to deal with that. It will retry a couple of times until it finally succeeds.
Could a failure in this mechanism be the reason for my zombies? (Happens with Wine 2.5, 2.6 and 2.10.)
BTW: As I saw, you replaced the "/tmp" with tempfile.gettempdir(), but this is actually not correct. Wineserver sockets will always be created in /tmp (hardcoded in the source) to ensure the correct socket is found, no matter which environment variables are set.
I did not expect that ... thanks.
Best regards, Sebastian
# Get full path of socket lock_path = os.path.join(self.server_path, 'lock')
# Status log self.log.out('[wine session] Expecting wineserver lock at %s ...' % lock_path)
# Status variable got_lock_pid = False # Time-step wait_for_seconds = 0.01 # Timeout timeout_after_seconds = 30.0 # Already waited for ... started_waiting_at = time.time() # Connection trys tried_this_many_times = 0
# Run loop until socket appears while True:
# Does socket file exist? if os.path.exists(lock_path):
# Count attempts tried_this_many_times += 1
# Open lock file lock_file = open(lock_path, 'rb') # Can I retrieve a PID? lock_pid = self.__get_lock_pid__(lock_file) # Close lock file lock_file.close()
# Check result if lock_pid is not None: got_lock_pid = True break
# Break the loop after timeout if time.time() >= (started_waiting_at + timeout_after_seconds): break
# Wait before trying again time.sleep(wait_for_seconds)
# Evaluate the result if not got_lock_pid:
self.log.out( '[wine session] ... was not locked %d (after %0.2f seconds & %d attempts)! Quit.' % ( timeout_after_seconds, tried_this_many_times ) ) sys.exit()
else:
# Log status self.log.out( '[wine session] ... is locked by PID %d (after %0.2f seconds & %d attempts)!' % ( lock_pid, time.time() - started_waiting_at, tried_this_many_times ) )
On 17.06.2017 12:20, Sebastian M. Ernst wrote:
@Bruno
Not sure if I got the problem you are describing exactly but what about doing the opposite direction? Make a very simple socket program for Windows that is launched and this program connects to your python application instead. When python notices the connection it is sure that wineserver is running because it just received the connection from an app running inside Wine. Also as long as this small utility is up wineserver won't close too.
Thanks, yes, I have considered this approach (and will likely follow it if everything else does not work). I'd prefer a "pythonic" solution without adding dependencies like a cross-compiler (for compiling the utility) to my module. I am already doing this for tests, but I'd really like to keep the module itself "clean" and easy to install. Just as a theoretical question, could such a utility be implemented as a batch file which wine (cmd) could handle "out of the box"?
@Sebastian
thanks again, this is really helpful.
If you disconnect before the real command is started the wineserver will assume it is no longer needed and terminate automatically. I would say this is a bug, because the user definitely does not expect that when passing "-p".
Is this worth opening a ticket?
I have sent a patch, lets see if it gets accepted. ;) -> https://source.winehq.org/patches/data/134955
Nevertheless, when you keep the connection open, it should actually work (and also does work for me, according to a quick test).
Once my wineserver is up, I try to do a number of path conversion using "winepath -w /some/path" etc. first before I fire up any exe-files. Am I correct in assuming that winepath should also use my wineserver (if run with the same prefix)?
Yes, it will use the same wineserver instance.
Can you check whether your quick test also works with winepath (eliminating a potential source of error)? I managed to turn winepath directly into a zombie a number of times by running winepath (too early) while wineserver was still starting. It does not produce any error messages whatsoever.
I have not been able to reproduce this problem, unfortunately. I have tested with the code which stays connected to the wineserver in order to workaround the issue above.
A slightly better solution which does not trigger this problem would be to wait for the wineserver lock (which will also appear in the server dir).
I used your code and implemented a routine which waits for the lock-file to be locked - see below this email. Usually, it requires two attempts / about 0.01 seconds until it succeeds. The PID returned by your code matches the PID of my wineserver. The problem is, just after I have confirmed the lock, winepath becomes a zombie again and again and ... occasionally, it works. Is this a bug or a lack of understanding on my part?
There is still a small race-condition (the lock is acquired but the socket has not been created yet), but Wine will be able to deal with that. It will retry a couple of times until it finally succeeds.
Could a failure in this mechanism be the reason for my zombies? (Happens with Wine 2.5, 2.6 and 2.10.)
For me it sounds like a similar problem as above, but I've also failed to reproduce this case. Of course there could be a bug somewhere in the Wine source, but I think this is rather unlikely.
Are you sure that the subprocess is not killed accidentally, for example? Or since you are using subprocess.PIPE for stdin/stdout/stderr, are the pipes maybe blocking execution of the program when they are not read periodically? For debugging purposes, you can also append '-d' to the wineserver args, which will print a message as soon as it is terminated.
Please note that if both methods work as expected, the second one has the advantage that there is no dummy connection which appears as regular process to the rest of Wine, so it would probably be preferred.
Best regards, Sebastian
-
Bruno Jesus -
Sebastian Lackner -
Sebastian M. Ernst