New SCSI Driver for Mac OS X
Hi, A while ago on this list, I was talking about implementing SCSI support on Mac OS X in Wine, and we all agreed the best way to do that would be to have a custom driver that provides an IOCTL interface. I've attached the driver (which is now finished) here. I've tested it on my system, and it works with this small test program that's also attached. But I want to make sure it's totally stable before I start working on the Wine side. So if you've got a Mac, and you're feeling brave, feel free to install my driver and run my test program. Extract the archive, then cd to the directory inside: tar jxf ExtMediaBSDClient.tar.bz2 cd ExtMediaBSDClient Install the extension to /Library/Extensions: cp -r ExtMediaBSDClient.kext /Library/Extensions If you're feeling really brave, you can install it to /System/Library/Extensions (and then it will be loaded on boot): cp -r ExtMediaBSDClient.kext /System/Library/Extensions The driver bundle has to be owned by root and belong to the wheel group, so make sure it does before trying to load it: sudo chown -R root:wheel /path/to/ExtMediaBSDClient.kext You can load the driver with the kextload command: kextload /path/to/ExtMediaBSDClient.kext You can verify that the driver loaded by running: kextstat | grep "ExtMediaBSDClient" Once it's loaded, all SCSI disks that are inserted afterward will accept the IOCTL. All disks that were inserted before won't however. To make sure a disk is using my driver, run the ioreg tool (it's part of Xcode) and look for "ExtMediaBSDClient": ioreg | grep "ExtMediaBSDClient" To test it, run the program 'ExtMediaTest' that I included. Pass it the filename of a device file for the disk (it should be one of the raw device files, e.g. /dev/rdisk1 instead of /dev/disk1) on which you want to test this: ./ExtMediaTest /path/to/device/file If it works, the program will print some of the INQUIRY data for that disk to the Terminal. If it doesn't work, it will print an error to the Terminal (and the driver will have logged something in the kernel log; see the Console app), or worse, the kernel will panic. (If it says "Inappropriate ioctl for device", that probably means it's not using my driver. Try ejecting the disk and reinserting it. And if it says the device is busy, try the program on one of the slices, e.g. /dev/rdisk1s1 instead of /dev/rdisk1.) If anything goes wrong (error messages, kernel panics, etc.), I want to know about it. If you get an error, paste the whole error message. If it's from ExtMediaTest, show me the lines in the kernel log that came from the driver (they have "ExtMediaBSDClient" or some such in them). If the kernel panics, attach the panic log. Also, tell me which version of Mac OS X you're using (so I know which KernelDebugKit to use). As soon as I verify that the driver works well, I'm going to start submitting patches to implement support for it in Wine. Chip
Excuse me, but why is this needed again? What IOCTL is missing in Mac OS X's native implementation? On Apr 27, 2010, at 2:10 PM, Charles Davis wrote:
Hi,
A while ago on this list, I was talking about implementing SCSI support on Mac OS X in Wine, and we all agreed the best way to do that would be to have a custom driver that provides an IOCTL interface.
I've attached the driver (which is now finished) here. I've tested it on my system, and it works with this small test program that's also attached. But I want to make sure it's totally stable before I start working on the Wine side. So if you've got a Mac, and you're feeling brave, feel free to install my driver and run my test program.
Extract the archive, then cd to the directory inside:
tar jxf ExtMediaBSDClient.tar.bz2 cd ExtMediaBSDClient
Install the extension to /Library/Extensions:
cp -r ExtMediaBSDClient.kext /Library/Extensions
If you're feeling really brave, you can install it to /System/Library/Extensions (and then it will be loaded on boot):
cp -r ExtMediaBSDClient.kext /System/Library/Extensions
The driver bundle has to be owned by root and belong to the wheel group, so make sure it does before trying to load it:
sudo chown -R root:wheel /path/to/ExtMediaBSDClient.kext
You can load the driver with the kextload command:
kextload /path/to/ExtMediaBSDClient.kext
You can verify that the driver loaded by running:
kextstat | grep "ExtMediaBSDClient"
Once it's loaded, all SCSI disks that are inserted afterward will accept the IOCTL. All disks that were inserted before won't however. To make sure a disk is using my driver, run the ioreg tool (it's part of Xcode) and look for "ExtMediaBSDClient":
ioreg | grep "ExtMediaBSDClient"
To test it, run the program 'ExtMediaTest' that I included. Pass it the filename of a device file for the disk (it should be one of the raw device files, e.g. /dev/rdisk1 instead of /dev/disk1) on which you want to test this:
./ExtMediaTest /path/to/device/file
If it works, the program will print some of the INQUIRY data for that disk to the Terminal. If it doesn't work, it will print an error to the Terminal (and the driver will have logged something in the kernel log; see the Console app), or worse, the kernel will panic. (If it says "Inappropriate ioctl for device", that probably means it's not using my driver. Try ejecting the disk and reinserting it. And if it says the device is busy, try the program on one of the slices, e.g. /dev/rdisk1s1 instead of /dev/rdisk1.)
If anything goes wrong (error messages, kernel panics, etc.), I want to know about it. If you get an error, paste the whole error message. If it's from ExtMediaTest, show me the lines in the kernel log that came from the driver (they have "ExtMediaBSDClient" or some such in them). If the kernel panics, attach the panic log. Also, tell me which version of Mac OS X you're using (so I know which KernelDebugKit to use).
As soon as I verify that the driver works well, I'm going to start submitting patches to implement support for it in Wine.
Chip <ExtMediaBSDClient.tar.bz2>
On 4/27/10 8:54 PM, C.W. Betts wrote:
Excuse me, but why is this needed again? What IOCTL is missing in Mac OS X's native implementation? Do I have to explain everything?!
All right. Here goes. Way back in October of last year, I started a discussion on this list about SCSI on Mac OS X: http://www.winehq.org/pipermail/wine-devel/2009-October/079257.html Then AJ objected to the first approach, saying that we really can't require the disk to be unmounted. This leaves option 2. This driver is part of an attempt to implement option 2. In summary: - There's no IOCTL for sending SCSI commands. - There's an API for sending them, but it's unrealistic for Wine. Therefore, we want to use an IOCTL, instead. - To implement an IOCTL, we need a driver. - This driver implements an IOCTL for sending SCSI commands. And don't suggest the SCSITask user client. I already did (as you can see in the linked post). Chip
On 4/28/10 4:03 AM, Stefan Dösinger wrote:
Am 27.04.2010 um 22:10 schrieb Charles Davis:
tar jxf ExtMediaBSDClient.tar.bz2 cd ExtMediaBSDClient Do you have the sourcecode somewhere with instructions on how to compile it?
Sure. I've attached the source here. To compile it, you need Xcode installed. After unpacking the tarball, from the XNUKernelExtras directory, just type: xcodebuild at a Terminal. Then to install it, do: sudo xcodebuild install DSTROOT=/ There's a header that goes along with it. To install it, you do: sudo xcodebuild -target headers install DSTROOT=/ I've also attached the source for the test program. You can just do: gcc -o ExtMediaTest ExtMediaTest.c on it after installing the header above. I don't have a public repository yet. My plan is to create a SourceForge project for this (like AJ suggested). Chip
Am 28.04.2010 um 17:35 schrieb Charles Davis:
On 4/28/10 4:03 AM, Stefan Dösinger wrote:
Am 27.04.2010 um 22:10 schrieb Charles Davis:
tar jxf ExtMediaBSDClient.tar.bz2 cd ExtMediaBSDClient Do you have the sourcecode somewhere with instructions on how to compile it?
Sure. I've attached the source here. To compile it, you need Xcode installed. Looks interesting. I prefer to have the sourcecode before I load something into my kernel :-)
Once it's loaded, all SCSI disks that are inserted afterward will accept the IOCTL. I assume that means I have to remove my cdrom from my drive and reinsert it after loading the driver, correct? Or does it mean removing the drive(as in unplugging and replugging a USB drive for example?)
On 4/28/10 2:14 PM, Stefan Dösinger wrote:
Am 28.04.2010 um 17:35 schrieb Charles Davis:
On 4/28/10 4:03 AM, Stefan Dösinger wrote:
Am 27.04.2010 um 22:10 schrieb Charles Davis:
tar jxf ExtMediaBSDClient.tar.bz2 cd ExtMediaBSDClient Do you have the sourcecode somewhere with instructions on how to compile it?
Sure. I've attached the source here. To compile it, you need Xcode installed. Looks interesting. I prefer to have the sourcecode before I load something into my kernel :-)
That's understandable.
Once it's loaded, all SCSI disks that are inserted afterward will accept the IOCTL. I assume that means I have to remove my cdrom from my drive and reinsert it after loading the driver, correct? Or does it mean removing the drive(as in unplugging and replugging a USB drive for example?)
If it's a removable disk drive, you don't have to physically unplug it. :) Reinserting the CD-ROM should work. Chip
On 4/28/10 9:35 AM, Charles Davis wrote:
I don't have a public repository yet. My plan is to create a SourceForge project for this (like AJ suggested). And here it is:
Hi Charles, Are the DKIOCPREVENT and DKIOCALLOW ioctls necessary? DiskArbitration already has a facility for clients to refuse mount/unmount/eject requests. http://developer.apple.com/mac/library/documentation/Darwin/Reference/DiscAr... Among other things, if Wine uses your driver to lock the media and then crashes, is there any way for the user to recover and unlock the media? Also, I don't know enough about the driver model to know if multiple user clients can compete or conflict for locking the media. For example, if two processes lock the media and only one unlocks it, is it locked or unlocked? DiskArbitration avoids these pitfalls. Also, at <http://xnu-extras.git.sourceforge.net/git/gitweb.cgi?p=xnu-extras/xnu-extras;a=blob;f=ExtMediaBSDClient.cpp;h=154dbd709476e682e86f14891b2fbc9fb9efa311;hb=HEAD#l154>, there's a mismatch between "path+6" and "pathlen-5", no? That's an argument for using strlcat, instead, isn't it? (By no means does the above mean I've done a thorough code review. It's just something I noticed.) Regards, Ken
On 4/28/10 5:02 PM, Ken Thomases wrote:
Hi Charles,
Are the DKIOCPREVENT and DKIOCALLOW ioctls necessary? They were the reason I first wrote the driver. I added DKIOCSCSICOMMAND later. DiskArbitration already has a facility for clients to refuse mount/unmount/eject requests.
http://developer.apple.com/mac/library/documentation/Darwin/Reference/DiscAr... That only works on requests that go through DiskArbitration. I'm pretty sure that won't stop a raw DKIOCEJECT (which by the way is what Wine uses). How can it, when DKIOCEJECT goes straight to the kernel?
Besides, to use it, we'd have to call DADiskMount() directly instead of going through diskutil. Which means we'd have to add some code to support DA to NTDLL. I don't know what AJ thinks of that.
Among other things, if Wine uses your driver to lock the media and then crashes, is there any way for the user to recover and unlock the media?
Just send the device file a DKIOCALLOW, and then everything will be back to normal.
Also, I don't know enough about the driver model to know if multiple user clients can compete or conflict for locking the media. For example, if two processes lock the media and only one unlocks it, is it locked or unlocked? DiskArbitration avoids these pitfalls. It's unlocked. It's like the old handles. 10 calls to HLock() would be undone by a single HUnlock(). I need to fix that. Luckily, the kernel gave me the process that initiated the (un)lock, so I can keep track of that. (Then I can also force the disk unlocked when it terminates.)
Also, at <http://xnu-extras.git.sourceforge.net/git/gitweb.cgi?p=xnu-extras/xnu-extras;a=blob;f=ExtMediaBSDClient.cpp;h=154dbd709476e682e86f14891b2fbc9fb9efa311;hb=HEAD#l154>, there's a mismatch between "path+6" and "pathlen-5", no? That's an argument for using strlcat, instead, isn't it? You're right. It's fixed now. (Personally, I'm surprised it didn't panic.)
Chip
On Apr 28, 2010, at 6:18 PM, Charles Davis wrote:
Besides, to use it, we'd have to call DADiskMount() directly instead of going through diskutil.
Huh? Who goes through diskutil? Who would have to call DADiskMount()? And why do you think DADiskMount() is necessary to register approval callbacks? That's not my understanding. One gets the DADiskRefs for the mounted disks via the disk-appeared callback.
Which means we'd have to add some code to support DA to NTDLL. I don't know what AJ thinks of that.
There's already DiskArbitration support in mountmgr.sys, added by Alexandre. :) At one point he tried using an unmount approval callback to have Wine close all of its handles for files on the device, but DA wouldn't even ask the callbacks if there were files open. I believe that has changed in Snow Leopard and we could revive that approach. That way, unmounting volumes would just work seamlessly without the need for the user to invoke eject.exe.so. But, anyway, there's no barrier to registering such an approval callback.
Among other things, if Wine uses your driver to lock the media and then crashes, is there any way for the user to recover and unlock the media? Just send the device file a DKIOCALLOW, and then everything will be back to normal.
OK, but that's not exactly something you can just tell a user to do.
Also, I don't know enough about the driver model to know if multiple user clients can compete or conflict for locking the media. For example, if two processes lock the media and only one unlocks it, is it locked or unlocked? DiskArbitration avoids these pitfalls. It's unlocked. It's like the old handles. 10 calls to HLock() would be undone by a single HUnlock(). I need to fix that. Luckily, the kernel gave me the process that initiated the (un)lock, so I can keep track of that. (Then I can also force the disk unlocked when it terminates.)
OK, although I don't know if you have an easy way to actually be told when a process terminates, or to link the lock to the process so it's automatically removed on process termination. -Ken
On 4/28/10 5:48 PM, Ken Thomases wrote:
On Apr 28, 2010, at 6:18 PM, Charles Davis wrote:
Besides, to use it, we'd have to call DADiskMount() directly instead of going through diskutil.
Huh? Who goes through diskutil? NTDLL, when it gets a FSCTL_DISMOUNT_VOLUME. Who would have to call DADiskMount()? The same. And why do you think DADiskMount() is necessary to register approval callbacks? That's not my understanding. One gets the DADiskRefs for the mounted disks via the disk-appeared callback. Sorry, I should have taken a closer look. I forgot about the approval callbacks.
Which means we'd have to add some code to support DA to NTDLL. I don't know what AJ thinks of that.
There's already DiskArbitration support in mountmgr.sys, added by Alexandre. :) I knew that.
At one point he tried using an unmount approval callback to have Wine close all of its handles for files on the device, but DA wouldn't even ask the callbacks if there were files open. Right, because it will refuse unless you tell it force-unmount. I believe that has changed in Snow Leopard and we could revive that approach. That way, unmounting volumes would just work seamlessly without the need for the user to invoke eject.exe.so. Cool. Maybe we should look into that.
But, anyway, there's no barrier to registering such an approval callback. Like I said, it will work when ejection and unmount requests go through DA, but not for raw umount(2) or ioctl(2) with DKIOCEJECT.
Also, I don't know enough about the driver model to know if multiple user clients can compete or conflict for locking the media. For example, if two processes lock the media and only one unlocks it, is it locked or unlocked? DiskArbitration avoids these pitfalls. It's unlocked. It's like the old handles. 10 calls to HLock() would be undone by a single HUnlock(). I need to fix that. Luckily, the kernel gave me the process that initiated the (un)lock, so I can keep track of that. (Then I can also force the disk unlocked when it terminates.)
OK, although I don't know if you have an easy way to actually be told when a process terminates, I don't, but I have *a* way:
The driver registers a kernel event socket (<sys/kern_event.h>) and kernel control socket (<sys/kern_control.h>). A daemon connects to both sockets. The daemon listens for events on the event socket. Every time the driver is told to lock the drive, it sends an event to the daemon. When the daemon gets this event, it adds the process it got to a list of processes for which it waits to die. When a process dies, the daemon sends a control message to the driver, and the driver unlocks the drive. Finally, every time the driver is told to unlock the drive, it sends another event to the daemon, which removes the process from the list.
or to link the lock to the process so it's automatically removed on process termination. Keep an OSSet of proc_t's that have locked the drive. Then when I get a note from the daemon, I unlock the drive.
-Ken
On Apr 28, 2010, at 8:28 PM, Charles Davis wrote:
On 4/28/10 5:48 PM, Ken Thomases wrote:
On Apr 28, 2010, at 6:18 PM, Charles Davis wrote:
Besides, to use it, we'd have to call DADiskMount() directly instead of going through diskutil.
Huh? Who goes through diskutil? NTDLL, when it gets a FSCTL_DISMOUNT_VOLUME.
Huh. Weird. I did not know that. I can't speak for Alexandre, but I suspect he would be fine using the appropriate Mac-native framework, whether that would be DiskArbitration or CoreServices (FSUnmountVolumeSync()), over invoking a Mac-specific command via system(). (So long as it's a C API, of course. ;)
But, anyway, there's no barrier to registering such an approval callback. Like I said, it will work when ejection and unmount requests go through DA, but not for raw umount(2) or ioctl(2) with DKIOCEJECT.
Is this something you've tested? I have not, but I would not be at all surprised if DiskArbitration had a kernel component. For example, if I mount a disk image via the Finder and then use umount(8) to unmount it, I get the relevant NSWorkspaceDidUnmountNotification. NSWorkspace is built on DiskArbitration's callbacks but umount(8) is not. It just uses the unmount(2) system call. (The above test was the quickest to hand. I didn't take the time to write a direct DiskArbitration test case, with or without an unmount approval callback.) -Ken
On 4/28/10 11:08 PM, Ken Thomases wrote:
On Apr 28, 2010, at 8:28 PM, Charles Davis wrote:
On 4/28/10 5:48 PM, Ken Thomases wrote:
On Apr 28, 2010, at 6:18 PM, Charles Davis wrote:
Besides, to use it, we'd have to call DADiskMount() directly instead of going through diskutil.
Huh? Who goes through diskutil? NTDLL, when it gets a FSCTL_DISMOUNT_VOLUME.
Huh. Weird. I did not know that.
I can't speak for Alexandre, but I suspect he would be fine using the appropriate Mac-native framework, whether that would be DiskArbitration or CoreServices (FSUnmountVolumeSync()), over invoking a Mac-specific command via system(). (So long as it's a C API, of course. ;) Don't know about that. The way it's implemented (and it's implemented by a function in dlls/ntdll/directory.c called 'DIR_unmount_volume), it invokes a command on both Mac OS X and other UNIXes/Linux. I think whoever implemented it used diskutil(8) on Mac OS X because it was the path of least resistance (given that it was already system(3)'ing umount(8) on other platforms).
But, anyway, there's no barrier to registering such an approval callback. Like I said, it will work when ejection and unmount requests go through DA, but not for raw umount(2) or ioctl(2) with DKIOCEJECT.
Is this something you've tested? I have not, but I would not be at all surprised if DiskArbitration had a kernel component.
For example, if I mount a disk image via the Finder and then use umount(8) to unmount it, I get the relevant NSWorkspaceDidUnmountNotification. NSWorkspace is built on DiskArbitration's callbacks but umount(8) is not. It just uses the unmount(2) system call. I just used a similar test case with a real disk, and I still got the DidUnmount note. But...
(The above test was the quickest to hand. I didn't take the time to write a direct DiskArbitration test case, with or without an unmount approval callback.) When I used DiskArbitration directly, and unmounted the disk with umount(8), it did not ask me if it was OK to unmount the disk. When I went through DA, however, I did get asked. I tried to test the same with DKIOCEJECT instead of unmount(2), but I couldn't open the whole media device file (got EBUSY).
It's probably true that AppKit has indeed registered callbacks with DA. I looked at its dependencies with otool, and DiskArbitration was one of them. But since it delivered the note when umount(8) was used and DA didn't, AppKit must therefore be using some other mechanism to discover this. (I thought it might be kqueues, but I looked at its imports with nm and couldn't find any references to kqueue(2) or kevent(2).) Chip
participants (4)
-
C.W. Betts -
Charles Davis -
Ken Thomases -
Stefan Dösinger