This resolution will make a ton of business apps based on desktop database software that much closer to working.
I'm not so sure. I expect most database software depends on the mandatory nature of LockFile(), i.e. that a ReadFile() or WriteFile() from another process on a locked region fails with ERROR_LOCK_VIOLATION. Typically, one process will LockFile() an area, then update on-disk structures in such a way that another process reading during the update would get confused were it not for ReadFile() refusing to read the locked bytes. Note that the reading process does not LockFile() to read, it just tries ReadFile() and handles its failure (typically by waiting and trying again later).
For database software that uses read/write locks via LockFileEx(), the standard Unix advisory lock semantics are just fine, and those patches are indeed the ticket. However, advisory locking is generally more difficult to manage, and LockFileEx() is not supported on Windows 95/98/Me, so I doubt many apps use it.
From what I can tell, mandatory locking on Linux involves tricks like using a
special mount option, and setting the SGID bit of the file to be locked before the locking process starts; I'm not sure which other Unixen even support it.
Hi John K. Hohm,
On Friday 21 March 2003 12:15 pm, John K. Hohm wrote:
This resolution will make a ton of business apps based on desktop database software that much closer to working.
I'm not so sure. I expect most database software depends on the mandatory nature of LockFile(), i.e. that a ReadFile() or WriteFile() from another process on a locked region fails with ERROR_LOCK_VIOLATION.
Yep, you seem to be correct.
Typically, one process will LockFile() an area, then update on-disk structures in such a way that another process reading during the update would get confused were it not for ReadFile() refusing to read the locked bytes. Note that the reading process does not LockFile() to read, it just tries ReadFile() and handles its failure (typically by waiting and trying again later).
For database software that uses read/write locks via LockFileEx(), the standard Unix advisory lock semantics are just fine, and those patches are indeed the ticket. However, advisory locking is generally more difficult to manage, and LockFileEx() is not supported on Windows 95/98/Me, so I doubt many apps use it.
I just confirmed... while locking seems to work if two instances of Visual Foxpro are running on Wine on the same machine, it DOES NOT work over the network. I had one instance on Wine and one instance on WinNT. Both instances were interacting with the same table on a third machine. The two instances of Visual FoxPro did not respect each other's locks. Bummer. Big bummer.
From what I can tell, mandatory locking on Linux involves tricks like using a special mount option, and setting the SGID bit of the file to be locked before the locking process starts; I'm not sure which other Unixen even support it.
So... chances of this ever getting implemented on Wine are, in your opinion, pretty slim? If so that's too bad, because I saw a great possibility of porting lots of legacy business apps over to Linux using Wine. There are, as I said before, a ton of apps out there built on desktop database software... bummer.
I've also run into the same situation - it works locally but not on samba shares. At first I though maybe wine server was just keeping its own table of locks, but I glanced through the code and it looks like its getting/setting the lock with the inode somehow (I'm pretty new to linux so I wouldn't know how that part works). It must be doing something about the advisory/mandatory problem because my read tests did fail on locked regions when performed on the same machine...
Regarding it not working on samba shares - I believe that samba just ignores unix locks because of the advisory/mandatory locking issue - which means that samba must be creating its own internal lock table. If this is the case, then it shouldn't be to difficult for wineserver to detect if the device is samba and map the lock requests through to samba when necessary (samba does have an api doesn't it?). I'll look into this when I get a chance - but maybe someone who is more familiar with the issue could crank it out pretty quickly?
- Eric Noel
----- Original Message ----- From: "Paul McNett" p@ulmcnett.com To: wine-devel@winehq.com Sent: Friday, March 21, 2003 4:34 PM Subject: Re: LockFile() and UnlockFile() are working
Hi John K. Hohm,
On Friday 21 March 2003 12:15 pm, John K. Hohm wrote:
This resolution will make a ton of business apps based
on desktop
database software that much closer to working.
I'm not so sure. I expect most database software
depends on the
mandatory nature of LockFile(), i.e. that a ReadFile()
or WriteFile()
from another process on a locked region fails with
ERROR_LOCK_VIOLATION.
Yep, you seem to be correct.
Typically, one process will LockFile() an area, then
update on-disk
structures in such a way that another process reading
during the update
would get confused were it not for ReadFile() refusing
to read the locked
bytes. Note that the reading process does not
LockFile() to read, it
just tries ReadFile() and handles its failure (typically
by waiting and
trying again later).
For database software that uses read/write locks via
LockFileEx(), the
standard Unix advisory lock semantics are just fine, and
those patches
are indeed the ticket. However, advisory locking is
generally more
difficult to manage, and LockFileEx() is not supported
on Windows
95/98/Me, so I doubt many apps use it.
I just confirmed... while locking seems to work if two
instances of Visual
Foxpro are running on Wine on the same machine, it DOES
NOT work over the
network. I had one instance on Wine and one instance on
WinNT. Both
instances were interacting with the same table on a third
machine. The two
instances of Visual FoxPro did not respect each other's
locks. Bummer.
Big bummer.
From what I can tell, mandatory locking on Linux
involves tricks like
using a special mount option, and setting the SGID bit
of the file to be
locked before the locking process starts; I'm not sure
which other Unixen
even support it.
So... chances of this ever getting implemented on Wine
are, in your opinion,
pretty slim? If so that's too bad, because I saw a great
possibility of
porting lots of legacy business apps over to Linux using
Wine. There are,
as I said before, a ton of apps out there built on desktop
database
software... bummer.
-- Paul McNett - p@ulmcnett.com Hollister, California, USA
On March 21, 2003 04:34 pm, Paul McNett wrote:
Hi John K. Hohm,
On Friday 21 March 2003 12:15 pm, John K. Hohm wrote:
This resolution will make a ton of business apps based on desktop database software that much closer to working.
snip
I just confirmed... while locking seems to work if two instances of Visual Foxpro are running on Wine on the same machine, it DOES NOT work over the network. I had one instance on Wine and one instance on WinNT. Both instances were interacting with the same table on a third machine. The two instances of Visual FoxPro did not respect each other's locks. Bummer. Big bummer.
Ah. And how were you accessing the third machine??? Were you, by any chance, using SMBFS?
I haven't looked at the locking yet but if Alexandre basically did it the same way as he did it last time (back in the early 90's I think) then the lock in wine should get passed on as a flock to the unix/linux, but last time I looked smbfs would not pass that on as a SMB lock request to the network.
That's why with our application we state that if there is a mix of Unix/Linux and Microsoft then you MUST host the locked file on a Unix/Linux platform, share between Unix/Linux using NFS (or something better if available) and share towards Microsoft using e.g. Samba.
snip
Paul McNett p@ulmcnett.com writes:
I just confirmed... while locking seems to work if two instances of Visual Foxpro are running on Wine on the same machine, it DOES NOT work over the network. I had one instance on Wine and one instance on WinNT. Both instances were interacting with the same table on a third machine. The two instances of Visual FoxPro did not respect each other's locks. Bummer. Big bummer.
AFAIK the Samba filesystem doesn't support locking. Locks should work across the network with NFS, but of course from a Windows machine that might be a problem. I'm afraid we can't do much in Wine, this will require fixing smbfs.
So... chances of this ever getting implemented on Wine are, in your opinion, pretty slim? If so that's too bad, because I saw a great possibility of porting lots of legacy business apps over to Linux using Wine. There are, as I said before, a ton of apps out there built on desktop database software... bummer.
Actually if you mount with the right options and set the right permissions on the file, mandatory locking should work. That's the theory though, I haven't tested it...
On March 21, 2003 04:59 pm, Alexandre Julliard wrote:
Paul McNett p@ulmcnett.com writes:
I just confirmed... while locking seems to work if two instances of Visual Foxpro are running on Wine on the same machine, it DOES NOT work over the network. I had one instance on Wine and one instance on WinNT. Both instances were interacting with the same table on a third machine. The two instances of Visual FoxPro did not respect each other's locks. Bummer. Big bummer.
AFAIK the Samba filesystem doesn't support locking.
Only the smbfs bit (which of course is not actually Samba)
Locks should work across the network with NFS, but of course from a Windows machine that might be a problem. I'm afraid we can't do much in Wine, this will require fixing smbfs.
The Samba server does the right thing, so host the file on unix/linux.
So... chances of this ever getting implemented on Wine are, in your opinion, pretty slim? If so that's too bad, because I saw a great possibility of porting lots of legacy business apps over to Linux using Wine. There are, as I said before, a ton of apps out there built on desktop database software... bummer.
Actually if you mount with the right options and set the right permissions on the file, mandatory locking should work. That's the theory though, I haven't tested it...
On Friday 21 March 2003 05:07 pm, Bill Medland wrote:
The Samba server does the right thing, so host the file on unix/linux.
Ok, let me get this straight. If I host the file on Linux using Samba, and connect to it using mixed Windows and Linux/Wine clients, the locking should be respected all around?
If so, then that's completely acceptable. For me, anyway.
So... where is the failing exactly? Windows SMB clients connecting to Windows SMB servers respect locks just fine. Windows SMB clients connecting to Linux Samba shares respect locks just fine (I've been doing this for years). Is the problem with the smbclient then? Sorry, I'm a layperson when it comes to all this lowlevel stuff...
Paul McNett p@ulmcnett.com writes:
So... where is the failing exactly? Windows SMB clients connecting to Windows SMB servers respect locks just fine. Windows SMB clients connecting to Linux Samba shares respect locks just fine (I've been doing this for years). Is the problem with the smbclient then?
Yes, the problem is that the smbfs Linux filesystem doesn't propagate locks. So you have to make sure that Wine is accessing the files through NFS, not through smbfs. As long as the server is a Linux machine this shouldn't be a problem.
On Friday 21 March 2003 05:52 pm, Alexandre Julliard wrote:
Yes, the problem is that the smbfs Linux filesystem doesn't propagate locks. So you have to make sure that Wine is accessing the files through NFS, not through smbfs. As long as the server is a Linux machine this shouldn't be a problem.
Ok I'm seeing better success with my new test, having the files on Linux/Samba and having mixed Windows and local Linux/Wine clients (Linux client is accessing local file directly).
However, it still isn't working 100%, or perhaps we've unearthed a new problem. Visual FoxPro's RLOCK() function is supposed to return TRUE if we are able to get a lock on the record (a block of the database file) and FALSE if not. Well, RLOCK() is still returning TRUE even if the other client already supposedly has the lock. This is the same behavior as before the recent implementation of LockFile() in Wine.
What is different, is that that once the second client (the one that never should have had RLOCK() return TRUE in the first place) tries to commit a change to the record, an Error Writing To File occurs. In normal operation on Windows, Error Writing To File rarely occurs, and definitely should never have occurred because we never should have gotten to the point where we were trying to write to a record that we never should have had the lock on.
So... my question to those that understand this stuff much better than I do... what is still missing? Is it ReadFile() that was mentioned by John K. Hohm earlier?
Paul McNett p@ulmcnett.com writes:
So... my question to those that understand this stuff much better than I do... what is still missing? Is it ReadFile() that was mentioned by John K. Hohm earlier?
It's quite possible that there are still bugs in the implementation. One thing to test would be with two clients running on the same machine, so that they use the same wineserver; this way all locking will be done internally without using Unix locks, and this might help identifying where the problem is.
Also if you can reproduce the bug easily I'd be very interested to see a +file,+server trace.
On Friday 21 March 2003 09:21 pm, Alexandre Julliard wrote:
It's quite possible that there are still bugs in the implementation. One thing to test would be with two clients running on the same machine, so that they use the same wineserver;
This seems to work correctly and consistently, although I admit I've only done rudimentary testing. But RLOCK() will correctly return TRUE or FALSE based on whether the other instance has the lock already or not, and we never reach the point that we get the "Error writing to file" because the lock was correctly detected from the outset.
this way all locking will be done internally without using Unix locks, and this might help identifying where the problem is.
Sorry for being thick (I'm sure this is a stupid question). Is there a way I can test on the same machine with two different wineservers (can I have more than one wineserver running on the same machine)? If possible, this would force Unix locks, right?
Also if you can reproduce the bug easily I'd be very interested to see a +file,+server trace.
Yes, bug reproduces easily. I'll do that tracing for you tomorrow. Thanks...
Paul McNett p@ulmcnett.com writes:
Sorry for being thick (I'm sure this is a stupid question). Is there a way I can test on the same machine with two different wineservers (can I have more than one wineserver running on the same machine)? If possible, this would force Unix locks, right?
You can easily have multiple wineservers, all you need is to run with a different WINEPREFIX. Simply copy your ~/.wine directory somewhere else, and then run one client with WINEPREFIX=~/.wine and the other with WINEPREFIX=~/some-other-place. This will then use Unix locks to synchronize between the two wineservers.
On Friday 21 March 2003 09:21 pm, Alexandre Julliard wrote:
It's quite possible that there are still bugs in the implementation. One thing to test would be with two clients running on the same machine, so that they use the same wineserver; this way all locking will be done internally without using Unix locks, and this might help identifying where the problem is.
Ok, I just did my simple RLOCK() testing using 2 wineservers on one local machine. The locking mechanism seemed to work perfectly using Unix locks. Both instances of Visual FoxPro respected one another's record locks. So it works locally... good! Presumably, this also would mean that the locks would be respected over NFS, although I have yet to test this.
However, hosting the database on Linux, accessed by Linux/Wine directly at /tmp/db.dbf and by Windows over samba, the behavior is not correct. It seems that instead of a range lock, a complete file lock is propogated, and it also seems that this file lock is never released.
Also if you can reproduce the bug easily I'd be very interested to see a +file,+server trace.
Ok:
http://www.paulmcnett.com/vfp/wine/lt1.gz
This shows the trace from the point of creating the dbf file (CREATE TABLE t:/locktest2 (cchar c(32))), closing it, reopening it, locking a record, and closing the app. Once the record was locked on the Linux side, my Windows instance of VFP couldn't write anywhere in the table. Once the record was unlocked on the Linux side, the Windows instance still couldn't write anywhere in the table. Basically, once locked, the windows side thought the entire file was read-only when in actuality only the one record should have been locked.
On Saturday 22 March 2003 12:15 pm, Paul McNett wrote:
However, hosting the database on Linux, accessed by Linux/Wine directly at /tmp/db.dbf and by Windows over samba, the behavior is not correct. It seems that instead of a range lock, a complete file lock is propogated, and it also seems that this file lock is never released.
Ok, I was either on drugs or too upset about this war my country is currently engaged in...
Before when I tested the Samba share, I didn't have the permissions set correctly... duh! Now, with correct permissions, files hosted on Samba and accessed by Windows do respect locks put in place by Wine, and vice-versa.
So, it appears that the only place this isn't working is when we mount a Samba share on Linux using smbfs. Which is what Alexandre was trying to get through my thick skull the whole time.
Alexandre, as far as I'm concerned this is working just fine as it appears this is a failing of smbfs and not at all a failing of Wine.
Thanks for the time.
Paul McNett wrote:
On Friday 21 March 2003 05:07 pm, Bill Medland wrote:
The Samba server does the right thing, so host the file on unix/linux.
Ok, let me get this straight. If I host the file on Linux using Samba, and connect to it using mixed Windows and Linux/Wine clients, the locking should be respected all around?
If so, then that's completely acceptable. For me, anyway.
So... where is the failing exactly? Windows SMB clients connecting to Windows SMB servers respect locks just fine. Windows SMB clients connecting to Linux Samba shares respect locks just fine (I've been doing this for years). Is the problem with the smbclient then? Sorry, I'm a layperson when it comes to all this lowlevel stuff...
That would seem to be the case. I saw a announcement has a new maintainer so maybe even this is not a long term problem
The smbfs filesystem The smbfs filesystem is a mountable SMB filesystem for Linux. It does not run on any other systems.
Up until recently smbfs has not been maintained as nobody in the Samba Team used it. This has now changed, and Andrew Tridgell tridge@linuxcare.com has taken over maintainence.
A new release of smbmount will be in the 2.0.6 release of Samba.
Samba 2.0.6 was released Mon Oct 21 16:11:25 2002
So.... Maybe we should droup him a line...
On Friday 21 March 2003 04:59 pm, Alexandre Julliard wrote:
Actually if you mount with the right options and set the right permissions on the file, mandatory locking should work. That's the theory though, I haven't tested it...
Ok I'll test this out. Anyone know what options I need to mount with?
Thanks!
-
Alexandre Julliard -
Bill Medland -
Eric -
John K. Hohm -
Paul McNett -
Tony Lambregts