Hi,
some days ago I've send a patch which broke some other tests, see http://www.kegel.com/wine/patchwatcher/results/274.txt . But this test got a "conformance test ok".
I think patchwatcher should also mark patches as bad when they let crash other tests.
Cheers Rico
On Fri, Aug 29, 2008 at 1:42 PM, Rico Schüller kgbricola@web.de wrote:
some days ago I've send a patch which broke some other tests, see http://www.kegel.com/wine/patchwatcher/results/274.txt . But this test got a "conformance test ok". I think patchwatcher should also mark patches as bad when they let crash other tests.
The full run log from that patch is at http://kegel.com/wine/patchwatcher/results/274.log Which of the test failures in there should have triggered a 'bad patch' result?
patchwatcher does a full 'make test' and looks for new test failures, so it's doing what you ask. The only wrinkle is that it has a blacklist of flaky tests, see http://code.google.com/p/winezeug/source/browse/trunk/patchwatcher/patchwatc... Is the test failure you're talking about on that blacklist? If so, patchwatcher will ignore it, as I've found the tests to be unreliable.
It's possible that the low-end nvidia motherboard graphics on the machine running patchwatcher contributed to this flakyness. I'm moving patchwatcher to a faster machine with a better nvidia card at the moment, perhaps that will let me shorten the blacklist. - Dan
Dan Kegel schrieb:
On Fri, Aug 29, 2008 at 1:42 PM, Rico Schüller kgbricola@web.de wrote:
some days ago I've send a patch which broke some other tests, see http://www.kegel.com/wine/patchwatcher/results/274.txt . But this test got a "conformance test ok". I think patchwatcher should also mark patches as bad when they let crash other tests.
The full run log from that patch is at http://kegel.com/wine/patchwatcher/results/274.log Which of the test failures in there should have triggered a 'bad patch' result?
patchwatcher does a full 'make test' and looks for new test failures, so it's doing what you ask. The only wrinkle is that it has a blacklist of flaky tests, see http://code.google.com/p/winezeug/source/browse/trunk/patchwatcher/patchwatc... Is the test failure you're talking about on that blacklist? If so, patchwatcher will ignore it, as I've found the tests to be unreliable.
It's possible that the low-end nvidia motherboard graphics on the machine running patchwatcher contributed to this flakyness. I'm moving patchwatcher to a faster machine with a better nvidia card at the moment, perhaps that will let me shorten the blacklist.
- Dan
My patch let the ddraw:d3d.c crash on wine (search for "0x60e52ad9 IWineD3DBaseSwapChainImpl_GetBackBuffer" in the log (http://kegel.com/wine/patchwatcher/results/274.log)). So it isn't in the blacklist and shouldn't be there. The problem here is that my patch breaks the test (the test crashes) but the patchwatcher didn't recognize it.
Cheers Rico
On Sat, Aug 30, 2008 at 2:13 AM, Rico Schüller kgbricola@web.de wrote:
My patch let the ddraw:d3d.c crash on wine (search for "0x60e52ad9 IWineD3DBaseSwapChainImpl_GetBackBuffer" in the log (http://kegel.com/wine/patchwatcher/results/274.log)). So it isn't in the blacklist and shouldn't be there. The problem here is that my patch breaks the test (the test crashes) but the patchwatcher didn't recognize it.
OK, looks like patchwatcher needs to also recognize lines like make[XXX]: *** [XXX.ok] Error XXX instead of relying on just messages from test.h. I'll try to fix that this weekend.
Thanks, Dan
-
Dan Kegel -
Rico Schüller