On Mon, 7 Mar 2016, Zhenbo Li wrote: [...]
So we can do
winetest -c -t <tag> -m <email> -o result_with_old_driver.txt d3dx9_36 d3d9 d3d8 etc. blah... winetest -c -t <tag> -m <email> -o result_with_new_driver.txt d3dx9_36 d3d9 d3d8 etc.
Then compare these two results with external tools(like diff or vimdiff)
Yes. The missing part is the comparison where it's mostly going to be diff. The TestBot's WineSendLog script has a CompareLogs() function to only report new failures to patch submitters (behind the scenes it uses Algorithm::Diff). Maybe that can be useful to you:
https://source.winehq.org/git/tools.git/blob/HEAD:/testbot/bin/WineSendLog.p...
With the script, he could type wine_test_suite --test_all --output result.txt And Bob can get everything he wants after his lunch.
winetest can do this.
Actually it's more that WineTest does this by default. But make can do it too: make -k test
[...]
Will it be too complicated if we allow developers to declare dynamic dependency manually? For example, I can add a line ## DEPENDENCY msxml3 in dlls/mshtml/Makefile.in to declare that mshtml.dll will depend on msxml3.dll dynamically.
Something like could be useful if the TestBot is to ever run the tests on Wine too. The alternative is to always run all the tests and either trust that they have zero failures by default anyway (which is the goal), or that a diff will find new ones.