On Tue, 5 Mar 2013, Dmitry Timoshkov wrote:
Francois Gouget fgouget@codeweavers.com wrote:
The goal of the Wine tests is to document the Windows behavior that Windows applications expect.
Skipping a test because your VM is broken doesn't qualify as a documentation of Windows behaviour.
The VM is not broken so the skip is ok (thanks for not bringing anything new to the table: it means I don't have to update my answer which makes this simpler).
A test is supposed to pass on a not broken Windows XP configuration, otherwise there wouldn't much point in creating the test.
Sure, let's camp. That's good because the VM is not broken so the skip is ok.
I don't think that you have resources and intention to have Windows VMs with all possible pre/post SP/hotfixes configurations.
I certainly intend to make it possible for Wine developers to run their tests on most significant Windows configurations and that includes each service pack and Internet Explorer version. It's not as resource intensive as you seem to think (*). Now that's different from the set of Windows configurations that every Wine patch will be run on. That will be a subset decided by the community.
My concern is not about computer resources, I'm sure you have plenty of spare CPU cycles to burn. The concern is about how far are you planning to go, and time it takes to manage. Microsoft releases hotfixes 1-2 times in a month, are you inteding to have VMs for every possible state of SPs+hotfixes?
Don't be silly, theer are hundred of hotfixes. As I already said, I intend to cover Service Packs, and Internet Explorer versions. I don't intend to provide arbitrary combinations of these either. What intend to do is follow a Windows machine's normal upgrade path. For instance for Windows XP my personal VM covers SP1 (+IE 6+WMP 8+DirectX 8), then adds SP2 (+.Net 1.0+WMP 9+DirectX 9.0c), then upgrades that to IE 7 (+.Net 1.1+WMP 10), then adds SP3 (+.Net 3.5+WMP 11) progression, then upgrades to IE 8 (+Silverlight) and finally has a last configuration with the more recent updates. Note that this last configuration is the only one that requires maintenance.
There are already some distinct things in each Windows VM to worry about (video, sound, locale, CD, etc.), so it becomes pretty critical to make wise decisions about what configurations you consider as major, and which of them you really want to manage.
Yes, each VM needs a good reliable foundation to build on. So particular attention must be given to the initial video, sound and it turns out network (with QEmu) hardware and driver configuraion. But that's independent of the number of test configurations you then add to the VM.
I consider the locale, CD and other variations to be independent from Windows version tests. So for instance all locale tests will likely be done on Windows 7 SP1 until that configuration is obsolete, which I expect will not be before a couple of years at least. I also intend to combine some tests to reduce the number of test configurations. For instance I don't think it would be an issue if the only configuration that has an Joliet+Rockridge CD is also the one that has two network cards and no sound card.