On 10/25/2017 06:57 PM, Francois Gouget wrote:
On Fri, 13 Oct 2017, Jeremy White wrote: [...]
A more reasonable ideal is that all of our tests would run successfully on a well curated list of 'rigorous' test machines. I don't believe that we have an official page for that; I maintain an unofficial one here: https://www.winehq.org/~jwhite/latest.html That is all of the 'newtb' Windows VMs excluding Windows 2000, Windows 8, and Windows 10.
I don't think it makes sense to exclude Windows 8 and 10 anymore. It's not like the early days where they had over 50 failures. Nowadays they barely have more failures than Windows 7.
I have updated my script in several ways: 1. I've included Windows 8 and Windows 10 2. I've added Linux results, although I include only one system (the 32 bit radeon) for now 3. I've made a distinction between 'usually fails' and 'sometimes fails'. We have a lot of sporadic failures where we accept that it's a transient network condition or acceptable race condition. This hopefully screens out the more important failures.
The result is that we now have 12 clearly wrong tests on Windows and 12 clearly wrong tests for Linux. The new Windows failures are mostly ddraw related, but the Linux failures are all over the map and might be easy pickings.
In an ideal world, we would drive all of the 'consistently failing' and 'usually failing' columns to zero.
Cheers,
Jeremy