On 03/10/2014 06:08 AM, Henri Verbeet wrote:
On 10 March 2014 03:13, Jeremy White jwhite@codeweavers.com wrote:
I wrote a script to analyze winetestbot results on all of the testbot vms except the win8 vms (they are just too broken to try to analyze right now).
I'm trying to get a handle on the nature of the bot failures; this current script looks for consistent failures (partly because a consistent failure that goes green is a win, and I badly want to track wins).
My results are here: http://www.winehq.org/~jwhite/ecd24b5a874e.html
Note that e.g. the win2000 testbot doesn't have results for all runs. It looks like this causes the script to classify some failures that should be "fixed" as intermittent failures. That might in turn cause someone to draw wrong conclusions about e.g. the ddraw tests, if they didn't pay enough attention to wine-patches.
Yes; not just win2k, but the win7u bot is unreliable, and one of the other win7 bots and one of the vista bots have a few drop outs as well.
But my code, in theory, skips holes in the data, so long as the data stays in line.
In other words, a pattern like this: SS-FFFF-FF where S is success, F is failure, and - is missing data, is considered 'fixed'. A pattern like this: F-F--F-F-F is considered 'consistently failing'. All other patterns are considered intermittent.
Note that it's only against the newtb vms; so you'll see the claim that d3d9:stateblock is fixed, but there is one non newtb machine where it still fails.
Cheers,
Jeremy