Glorious winetestbot results, why this matters
So here's today's data: http://www.winehq.org/~jwhite/31faed571d5a.html The main changes are the addition of some new failures in mshtml; failures that Marvin actually pointed out here: http://www.winehq.org/pipermail/wine-devel/2014-March/103419.html The glorious result is that if I get to publicly shame anyone, it's both Jacek and Alexandre. For someone with my JR, that's a rare opportunity <grin>. But really, it illustrates the problem: Alexandre does not trust the new test bot results, so he did not wait for testbot review of patches, and so he did not catch a problem. We need to restore that trust. That's why this matters. Cheers, Jeremy p.s. Looks like Francois's patch has fixed winmm:mci (yay!), and a few tests that my script felt were 'solved' have proven themselves to be actually intermittent (comctl:propsheet and user32:listbox)
On 12 March 2014 15:31, Jeremy White <jwhite(a)codeweavers.com> wrote:
But really, it illustrates the problem: Alexandre does not trust the new test bot results, so he did not wait for testbot review of patches, and so he did not catch a problem. We need to restore that trust. That's why this matters.
I'm not going to argue about the basic point, but I don't think it's all that clear that some of the tests timing out on occasion is really about the tests themselves, as opposed to the testbot infrastructure.
On 03/12/2014 09:48 AM, Henri Verbeet wrote:
On 12 March 2014 15:31, Jeremy White <jwhite(a)codeweavers.com> wrote:
But really, it illustrates the problem: Alexandre does not trust the new test bot results, so he did not wait for testbot review of patches, and so he did not catch a problem. We need to restore that trust. That's why this matters.
I'm not going to argue about the basic point, but I don't think it's all that clear that some of the tests timing out on occasion is really about the tests themselves, as opposed to the testbot infrastructure.
I asked Alexandre, and he thought that this one might be a fair cop; not just an infrastructure failure, but a real failure that should have been caught. But you are right that the infrastructure instability makes it very difficult to do any real work against the failing test list. My hope is that it's close enough now that we can, by trying to fix tests, more clearly identify the remaining issues and resolve them. And I also felt remiss in failing to shame *anyone* <grin>. Cheers, Jeremy
Jeremy White <jwhite(a)codeweavers.com> writes:
On 03/12/2014 09:48 AM, Henri Verbeet wrote:
On 12 March 2014 15:31, Jeremy White <jwhite(a)codeweavers.com> wrote:
But really, it illustrates the problem: Alexandre does not trust the new test bot results, so he did not wait for testbot review of patches, and so he did not catch a problem. We need to restore that trust. That's why this matters.
I'm not going to argue about the basic point, but I don't think it's all that clear that some of the tests timing out on occasion is really about the tests themselves, as opposed to the testbot infrastructure.
I asked Alexandre, and he thought that this one might be a fair cop; not just an infrastructure failure, but a real failure that should have been caught.
It was caught, it's just that there are spurious timeouts often enough that I assumed it was probably a testbot glitch, and committed the patch so that we'd find out if it was real... -- Alexandre Julliard julliard(a)winehq.org
On 03/12/14 15:31, Jeremy White wrote:
So here's today's data: http://www.winehq.org/~jwhite/31faed571d5a.html
The main changes are the addition of some new failures in mshtml; failures that Marvin actually pointed out here: http://www.winehq.org/pipermail/wine-devel/2014-March/103419.html
The glorious result is that if I get to publicly shame anyone, it's both Jacek and Alexandre. For someone with my JR, that's a rare opportunity <grin>.
This should be fixed by my recent patches. The more worrying failure is crypt32:chain. Those happened with no change in tests at all and they don't happen on single test runs: https://testbot.winehq.org/JobDetails.pl?Key=5623 https://testbot.winehq.org/JobDetails.pl?Key=5624 I sent a patch that should help diagnose the problem if failures will continue to happen on full test runs. Cheers, Jacek
On Wed, Mar 12, 2014 at 10:51 AM, Jacek Caban <jacek(a)codeweavers.com> wrote:
On 03/12/14 15:31, Jeremy White wrote:
So here's today's data: http://www.winehq.org/~jwhite/31faed571d5a.html
The main changes are the addition of some new failures in mshtml; failures that Marvin actually pointed out here: http://www.winehq.org/pipermail/wine-devel/2014-March/103419.html
The glorious result is that if I get to publicly shame anyone, it's both Jacek and Alexandre. For someone with my JR, that's a rare opportunity <grin>.
This should be fixed by my recent patches. The more worrying failure is crypt32:chain. Those happened with no change in tests at all and they don't happen on single test runs:
https://testbot.winehq.org/JobDetails.pl?Key=5623 https://testbot.winehq.org/JobDetails.pl?Key=5624
I sent a patch that should help diagnose the problem if failures will continue to happen on full test runs.
Thanks, Jacek. That one's an oddball test failure. Random bad guess: certificate chain caching, including caching the chain verification results? Anyway, looking forward to getting more data on it. --Juan
participants (5)
-
Alexandre Julliard -
Henri Verbeet -
Jacek Caban -
Jeremy White -
Juan Lang