Module: tools Branch: master Commit: a26e789fad7cc0486b6e967ad97b4113082aa24c URL: https://source.winehq.org/git/tools.git/?a=commit;h=a26e789fad7cc0486b6e967a...
Author: Francois Gouget fgouget@codeweavers.com Date: Mon Feb 8 15:21:56 2021 +0100
testbot/reporttest: Match the tests to the wtbsuite.
Where a test mirrors a wtbsuite test, add the latter's commit message as a comment to make it simpler to compare the two. Also tweak some comments to better describe the tests.
Signed-off-by: Francois Gouget fgouget@codeweavers.com Signed-off-by: Alexandre Julliard julliard@winehq.org
---
testbot/src/reporttest/report.template | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-)
diff --git a/testbot/src/reporttest/report.template b/testbot/src/reporttest/report.template index c08b4d4..8da3d82 100644 --- a/testbot/src/reporttest/report.template +++ b/testbot/src/reporttest/report.template @@ -209,8 +209,11 @@ Dll info: xmllite=1.0.1018.0 Test output: stub advapi32:cred +----- WTBS Simple test patch +----- Expected assessement: Success
advapi32:crypt start dlls/advapi32/tests/crypt.c - +----- WTBS Simple test failure ----- A test unit with one failed test ----- Expected assessement: 1 failure This also has a valid source file name which one can link to. @@ -219,10 +222,11 @@ crypt.c:1: Test failed: Something wrong advapi32:crypt:031c done (1) in 0s
advapi32:crypt_lmhash start dlls/advapi32/tests/crypt_lmhash.c - +----- WTBS Simple diacritic failure ----- A test unit with two failed tests ------ Expected assessement: 2 failures +----- Expected assessement: 2 failures despite exit code crypt_lmhash.c:10: Test failed: Something wrong -crypt_lmhash.c:20: Test failed: Second failure here +crypt_lmhash.c:20: Test failed: Second failure with added diacritics: ?????????? 07c0:crypt_lmhash: 95 tests executed (0 marked as todo, 2 failures), 0 skipped. advapi32:crypt_lmhash:07c0 done (1) in 0s
@@ -246,7 +250,8 @@ Note that this also applies to the todo and skip counts. advapi32:crypt_md5:087c done (3) in 0s
advapi32:crypt_sha start dlls/advapi32/tests/crypt_sha.c - ------ A test unit with a failed subprocess test ignored by the parent +----- A test unit with a failed subprocess test ignored by the parent test +----- summary but accounted for in the exit code. ----- Expected assessement: 1 failure crypt_sha.c:20: Test failed: A failure here, presumably in the subprocess 0456:crypt_sha: 1 tests executed (0 marked as todo, 1 failure), 0 skipped. @@ -256,6 +261,7 @@ subprocess. advapi32:crypt_sha:08f8 done (3) in 0s
advapi32:eventlog start dlls/advapi32/tests/eventlog.c - +----- WTBS Unreported test failures ----- A test unit with an unreported failed test ----- Expected assessement: 1 failure, "Unreported failure" error eventlog.c:20: Test failed: Something wrong @@ -263,6 +269,7 @@ eventlog.c:20: Test failed: Something wrong advapi32:eventlog:088c done (3) in 4s
advapi32:lsa start dlls/advapi32/tests/lsa.c - +----- WTBS Missing test summary line ----- A test unit with no test summary line ----- Expected assessement: "Missing test summary" error advapi32:lsa:083c done (0) in 0s @@ -308,6 +315,7 @@ files.c:20: Test failed: A failure in the main process advpack:files:0880 done (3) in 0s
advpack:install start dlls/advpack/tests/install.c - +----- WTBS Simple test crash ----- A test unit with a crash ----- Expected assessement: Crash 07b0:install: unhandled exception c0000005 at 0040167C @@ -371,6 +379,8 @@ comcat:comcat start dlls/comcat/tests/comcat.c - comcat:comcat:0948 done (0) in 0s
comctl32:animate start dlls/comctl32/tests/animate.c - +----- WTBS Timeout in a test +----- WTBS Timeout in a test - barely ----- A test unit with a timeout ----- Expected assessement: Timeout comctl32:animate:0b14 done (258) in 42s @@ -380,7 +390,10 @@ comctl32:button skipped dlls/comctl32/tests/button.c - ----- Expected assessement: Skipped by user
comctl32:comboex start dlls/comctl32/tests/comboex.c - ------ A test unit with failed, todo and skipped tests in the main process +----- WTBS Simple success in a todo test +----- WTBS Simple trace, ok, todo, skip sampler +----- A test unit with a subprocess and failed, todo and skipped tests in the +----- main process. ----- Expected assessement: 2 failures, 1 todo, 1 skip comboex.c:40: Test failed: A failure in the main process comboex.c:41: Test succeeded inside todo block: Success in the main process @@ -394,6 +407,7 @@ comctl32:comboex:0c9c done (2) in 0s comctl32:datetime start dlls/comctl32/tests/datetime.c - ----- A test unit reporting a failure but with no test failed line ----- Expected assessement: "Missing test failed line" error +----- + TestFailures=2 (1 for the exit code + 1 for the missing failure line) 0998:datetime: 651 tests executed (0 marked as todo, 1 failures), 0 skipped. comctl32:datetime:0998 done (1) in 0s
@@ -425,8 +439,10 @@ comctl32:ipaddress start dlls/comctl32/tests/ipaddress.c - Issued upon seeing the next start line. 0998:ipaddress: 16 tests executed (0 marked as todo, 0 failure), 0 skipped.
+----- Keep this stub in case the previous test is mishandled stub comctl32:listview
+----- WTBS A test unit with no start line ----- A test unit with a missing start line ----- Expected assessement: "Missing start line" error 0998:misc: 16 tests executed (0 marked as todo, 0 failure), 0 skipped. @@ -490,6 +506,7 @@ crashed process which makes sense since it did not crash. comctl32:subclass:07b0 done (1) in 0s
comctl32:syslink start dlls/comctl32/tests/syslink.c - +----- WTBS Garbled trace, ok, todo, skip sampler ----- A test unit with garbled test lines ----- Expected assessement: 2 failures, 1 todo, 1 skip Multi-threaded and multi-process tests should really try to avoid races in @@ -518,13 +535,14 @@ comctl32:toolbar done (-1073741819) in 0s stub comctl32:tooltips
comctl32:trackbar start dlls/comctl32/tests/trackbar.c - +----- WTBS Test failures, todos and skips in a subtest ----- A test unit with a subtest +----- Expected assessement: 2 failures, 1 todo, 1 skip trackbar.c:496: Subtest driver driver.c:40: Test failed: Something wrong driver.c:41: Test succeeded inside todo block: Something right! driver.c:42: Test marked todo: Hello Wine! driver.c:43: Tests skipped: Another time ------ Expected assessement: 2 failures, 1 todo, 1 skip 07b1:trackbar: 3 tests executed (1 marked as todo, 2 failures), 1 skipped. comctl32:trackbar:07b1 done (2) in 0s
@@ -1011,7 +1029,7 @@ stub xcopy.exe:xcopy stub xmllite:reader
xmllite:writer start dlls/xmllite/tests/writer.c - ------ A test unit with a misplaced test failed line +----- A test unit with a misplaced (late) test failed line ----- Expected assessement: "Misplaced test failed line" error See the similar scenario before but note the lack of the extra test summary line here. Also beware of the end-of-file.