Hello everyone, I was considering writing a few regression tests for wine, and after reading Francois' presentation, I thought perhaps a File I/O would be a good one to implement (especially since he mentions CreeateFile). So I began thinking about how to go about it, which led me to the following...
Has any thought been given to creating a sandbox for regression tests? For instance, let's look at creating a test for File I/O. This requires creating/deleting files/directries, the ability to locate files, etc.
But I don't see anything in the framework that lets me do this portably on all systems. In wine, I can't gaurantee that the test directory (which I could probably write to) is mapped to a Windows drive. On all platformes, I don't see how I can write to an arbitrary location on C:, since it may be unwriteable, and even if it's not, people probably don't want extraneous wine stuff littered around their drive when something goes wrong.
I'm now thinking I'll try something more straight-forward as my first attempt, but this type of test seems like it'd be very useful, and I don't see how to do it with today's infrastructure.
Ideas? Thanks, .Geoff
----------------------------------------------------- Protect yourself from spam, use http://sneakemail.com
On 23 Mar 2002, Geoffrey Hausheer wrote:
Hello everyone, I was considering writing a few regression tests for wine, and after reading Francois' presentation, I thought
I'm glad my presentation has inspired you :-)
[...]
But I don't see anything in the framework that lets me do this portably on all systems.
To portably create files and directories on all system just use CreateFile & co.
In wine, I can't gaurantee that the test directory (which I could probably write to) is mapped to a Windows drive. On all platformes, I don't see how I can write to an arbitrary location on C:, since it may be unwriteable, and even if it's not, people probably don't want extraneous wine stuff littered around their drive when something goes wrong.
I believe most file tests don't need to write to arbitrary locations on the C:\ drive. For such tests, the best is probably to change the current directory to $Temp (should be defined on Windows too) and create files and directories there. Some other tests will need a writable c:\ drive... or other. Maybe tests should check a WINETEST_DRIVE environment variable.
In any case, tests should cleanup after themselves, i.e. call DeleteFile after they are done with a file, and be able to cope in case they were abruptly terminated, i.e. not fail if the file already exists, or Delete it before hand.
We may be able to add some infrastructure to handle this but doing so is not easy because it is full of compromises: * we want the infrastructure to be simple. Otherwise this makes writing tests complex which is bad. * a simple infrastructure is likely to only handle simple cases well. But simple cases are those where we need the infrastructure the least. So the infrastructure would have to be really easy to make even these cases even easier. * to handle the more complex cases the infrastructure woul dhave to be more complex. And that's bad as per point 1. * if each test does things its own way, things will be done slightly differently each time. This is actually good if it means more coverage of APIs. It's bad if it becomes a maintenance nightmare or leads to too many poorly written tests.
To summarise, I think it's not clear at this time exactly what the infrastructure should do for testing files. I believe the best thing to do is to write a few of the file tests, including some of the hard ones like those that convert paths. Then, when we have half a dozen or more we can look back and see if something can be done to make them simpler (for the next hundred ;-).
-- Francois Gouget fgouget@free.fr http://fgouget.free.fr/ Avoid the Gates of Hell - use Linux.
-
Francois Gouget -
Geoffrey Hausheer