On Sat, 27 Oct 2007, Stefan Dösinger wrote: [...]
If you look at various game boxes in the software shop next to you, you will very often find notes like "Works on ATI radeon and Nvidia geforce cards. Mobile versions of the cards are unsupported". Means, that if you have a Destop radeon or gf card, you're lucky. If you have a mobile radeon or gf card, and the game doesn't work, your problem. If you have an Intel card, don't even try to start the game.
If the game fails on Windows, users will either blame the game maker or Intel. We don't care. However if it fails in Wine, the user will blame Wine, not Intel, and not the game maker.
[...]
I don't think the caps will really be useful here, but we could retrieve the card and driver name, and skip the test for some combinations. IDirect3D9::GetAdapterIdentifier() for example should return enough information for d3d9. I'm not sure we want to go there though, Alexandre has said before that the tests should simply fail on broken setups.
Was he talking about broken Windows setups or broken Linux setups? It's find to ignore clearly broken Linux setups (e.g. known buggy driver version), but calling every Windows system that does not give the result we want 'broken' is refusing to face reality.
When FindExecutable() returns success on NT4 instead of returning an error like it should, we don't say that NT4 is broken and that thus it's ok to let the test fail and that it is its job to fail. We don't do that because the test's job is not to detect bugs in Windows, but to document what the different versions of Windows do, and to make sure that Wine does the same as one of them. So we adjust the test to accept both results as valid so it succeeds on both systems.
I don't see why it should be different for the DirectX tests.
Yes, that will work to filter out vmware, but I don't think we should do that. The test just does its job when it fails on broken drivers.
Its job is to needlessly pollute the results?
Again, the test's job is not to detect broken systems, but to document the Windows behavior.
It seems what you want is unit tests that would garantee that WineD3D behaves in the way which _you_ have decided. Most of the time that's the same as writing a _conformance_ test, but apparently in this specific case it's different. If that's so, then maybe we have to see how we can distinguish the two (separate test suites, enabling additional tests when running in Wine, etc).
This sort of infrastructure could also be useful in the case I mentionned above, where a specific version of Windows returns something stupid, but where we'd very much prefer to match the behavior of the other non-buggy versions.