Test units most frequently getting new failures: grep -v -P '^(#| +[0-9])' failures-new.txt | sed -n '/week 2023-10-16$/,/week 2023-08-28$/p' | grep -E -v '[-]> fixed' | sed -e 's/[^a-z0-9:._]/\n/g' | grep -E '[a-z0-9]:[a-z0-9]' | sort | uniq -c | sort -n -r # or to filter on days instead of weeks sed -n '/^2023-10-13/,/^2023-10-02/p' | # or to exclude fixed issues sed -e 's/[a-z][a-z0-9:._]*:[a-z][a-z0-9:._]* -> fixed.*//g' | # or to deduplicate origins that cause multiple bugs sed -e 's/^\(.. [^ ]*\) .*/\1/' -e t -e d | sort -u | # or to filter out failures caused by GitLab's switch to Debian 12 grep -E -v 5799e89f15b5 | # or to keep only the data lines grep -E '^.. ' | sort | # Origin counts # <...> main grep, time period and other filters go here cat >/tmp/r$$.txt; for t in r c m "."; do echo $t $(grep -E -i "^$t. " /tmp/r$$.txt | wc -l); done; rm /tmp/r$$.txt # or reason counts cat >/tmp/r$$.txt; for t in "[.]" b c m e o f g h i l w r t "[?]" "."; do echo $t $(grep -E -i "^.$t " /tmp/r$$.txt | wc -l); done; rm /tmp/r$$.txt # All bug numbers sed -e 's/\(bug [0-9][0-9]*\)/\n\1\n/g' | sed -e 's/^bug //' -e t -e d | sort -u | wc -l # or to filter out fixed bugs prefix with grep -E -v 'fixed bug' | For importing in spreadsheets: (p=2099-12-31; for d in `sed -e 's/ ---.*//' failures-new.txt | grep -P '^[0-9]{4}-[0-9]{2}-[0-9]{2}$'` ; do echo -n "$p,"; for t in "[.]" b c m e o f g h i l w r t "[?]" "."; do count=`sed -n "/^$p/,/^$d/p" failures-new.txt | grep -E -i "^.$t " | wc -l`; echo -n "$count,"; done; p=$d; echo; done) | tac Jul grep -v -P '^(#| +[0-9])' failures-new.txt | sed -n '/^2023-07-28/,/^2023-06-30/p' | Aug grep -v -P '^(#| +[0-9])' failures-new.txt | sed -n '/^2023-08-28/,/^2023-07-28/p' | Sep grep -v -P '^(#| +[0-9])' failures-new.txt | sed -n '/^2023-10-06/,/^2023-08-28/p' | # One character classification codes: # # ORIGINS : Where does the new failure come from? # # r Unexplained new failure # The reason why this new failure appeared is not known. It could be a # change in Wine, making it a regression, or some unidentified environment # change or glitch. Even in that case it could mean the test is buggy in # that it is not robust enough. # c Commit causing a regression # The new failure is caused by a commit which went neither through the # TestBot nor through the GitLab CI. # m Merge request causing a regression # The new failure is caused by the specified merge request. # # REASONS : Why did the TestBot not detect it? # # . Untested # No tests were performed prior to the regression being introduced in Wine # (i.e. this can only be paired with m or c origins). # This is typically the case for commits that don't go through GitLab but it # can also happen if an MR is modified during the merge. # b Bad merge # The failures were detected and reported but the merge request was merged # anyway. # c GitLab CI only # Happens in the GitLab CI but not on the TestBot VMs. # m Module patch # The patch does not touch the tests, causing the TestBot to skip running # the tests in some configurations, most notably Windows, and thus not # detect the regression. # e Extra configuration # The failure could not be detected by the TestBot because it only happens # in one of the non-base TestBot configurations. # o Other configuration # The failure only happens in non-TestBot / GitLab test configurations such # as running the 32-bit tests in the same wineprefix after the 64-bit tests. # f Flaky # The failure did not happen when the TestBot tested the merge request # despite the relevant configuration being tested and the failure happening # outside WineTest runs, presumably because it happens randomly and rarely # enough. # g Git skew # The patch was meant to be apply on a top newer Git revision but the # TestBot received it before it noticed that Git revision was available, # leading to a bad patch error. # h Hidden # Some other failure or issue hid the new failure. For instance this can # happen when network trouble prevents some tests from running correctly or # if a patch fails to apply or to compile. This can also happen when two MRs # succeed separately but interfere and cause failures when combined. # i Minor # The new failure is minor and was ignored by the TestBot. This is typically # the case of commits that introduce dependencies on missing APIs. # l Linux test suite # The failure only happen when running the full test suite in Wine. # w Windows test suite # The failure only happen when running the full test suite on Windows. # r Report bug # The failure was detected by the TestBot but not reported to GitLab. This # should only be used after having ferified that the TestBot did detect the # failure. # t TestBot bug # The failure was not detected because of a TestBot bug: it did not detect # the failure, thought it was a preexisting one, or it did not run the right # tests. # ? Status to be determined DONE: 55128 (2023-06-24) -> 55807 (2023-10-18) TODO: Bugs before 54659 (2023-03-14) 2023-10-20 --- week 2023-10-16 2023-10-13 --- week 2023-10-09 2023-10-10 me 4024 uiautomationcore:uiautomation -> bug 55807 mr 3959 uxtheme:system -> bug 55771 mr 3959 uxtheme:system -> bug 55772 2023-10-09 ro unknown psapi:psapi_main -> bug 55803 (rb-wayland) 2023-10-06 --- week 2023-10-02 c. de81e2ea41b2 dbghelp:dbghelp -> bug 55747 c. de81e2ea41b2 ntdll:loader -> bug 55746 c. de81e2ea41b2 ntdll:wow64 -> bug 55755 c. de81e2ea41b2 psapi:psapi_main -> bug 55744 c. de81e2ea41b2 tasklist.exe:tasklist -> bug 55745 r? unknown wbemprox:query -> bug 55751 (rb-macos) 2023-10-05 me 4009 advapi32:eventlog -> bug 55731 c. 4847c1d8e474 kernel32:loader -> fixed bug 55732 me 4020 ntdll:file -> bug 55733 2023-10-03 r? unknown amstream:amstream -> bug 55725 r? unknown d2d1:d2d1 -> bug 55765 mc 4004 msxml3:domdoc -> bug 55720 2023-10-02 r? unknown mfmediaengine:mfmediaengine -> bug 55724 2023-09-29 --- week 2023-09-25 mo 3953 dmime:dmime -> bug 55722 (rb-macos) ro unknown rpcrt4:rpc -> bug 55689 ro unknown winhttp:winhttp -> bug 55690 2023-09-28 c. a81c53504ae3 msvfw32:mciwnd -> fixed bug 55696 r? unknown ntdll:info -> bug 55721 2023-09-27 ro unknown user32:win -> bug 55767 (rb-wayland) 2023-09-26 ro unknown kernel32:file -> bug 55668 (rb-macos) ro unknown kernel32:file -> bug 55568 (rb-macos) 2023-09-25 mr 3930 d3d10:effect -> fixed bug 55651 r? unknown dmime:dmime -> bug 55687 c. 0abd795ff026 kernel32:file -> bug 55667 (rb-macos) 2023-09-22 --- week 2023-09-18 2023-09-19 mc 3882 msvcr120:msvcr120 -> fixed bug 55686 mr 3836 ntdll:file -> fixed bug 55631 mr 3836 ntdll:file -> fixed bug 55632 2023-09-18 ro unknown dbghelp:dbghelp -> fixed bug 55650 (rb-macos) mr 3867 mfreadwrite:mfplat -> fixed bug 55635 2023-09-15 --- week 2023-09-11 2023-09-14 ro unknown ntdll:wow64 -> fixed bug 55648 ro unknown psapi:psapi_main -> fixed bug 55658 (rb-macos) mr 3832 user32:dialog -> fixed bug 55656 mr 3832 user32:dialog -> bug 55655 2023-09-12 mr 3792 quartz:vmr9 -> fixed bug 55587 mr 3792 quartz:vmr9 -> bug 55649 2023-09-11 mf 3751 dmime:dmime -> bug 55637 ro unknown ntdll:exception -> bug 55657 (rb-macos) 2023-09-08 -- week 2023-09-04 mr 3746 d3d8:device -> bug 55583 2023-09-07 c. 425bd55b02e3 dbghelp:dbghelp -> fixed bug 55560 2023-09-06 mf 3623 uiautomationcore:uiautomation -> fixed bug 55580 2023-09-05 m. 3643 combase:roapi -> fixed bug 55531 (AJ merge) m. 3643 kernel32:actctx -> fixed bug 55532 (AJ merge) m. 3643 setupapi:devinst -> fixed bug 55533 (AJ merge) m. 3643 setupapi:install -> fixed bug 55533 (AJ merge) m. 3643 ucrtbase:thread -> fixed bug 55534 (AJ merge) mt 3548 windows.applicationmodel:model -> fixed bug 55652 (missing makedep testdll support) 2023-09-04 r? unknown advapi32:registry -> fixed bug 55577 c. d64200805b18 kernel32:loader -> bug 55578 2023-09-01 -- week 2023-08-28 c. d64200805b18 kernel32:loader -> bug 55579 2023-08-31 mr 3691 dinput:hid -> fixed bug 55535 2023-08-30 mm 3681 mshtml:dom -> bug 55569 ro unknown quartz:mpegaudio -> bug 55552 (rb-macos) ro unknown quartz:mpeglayer3 -> bug 55553 (rb-macos) 2023-08-28 c. f6dbcccab9e3 kernel32:virtual -> bug 55561 2023-08-25 --- week 2023-08-21 2023-08-23 m? 3439 mf:mf -> bug 55588 2023-08-18 --- week 2023-08-14 2023-08-11 --- week 2023-08-07 2023-08-08 r? unknown quartz:filtergraph -> bug 55758 r? unknown user32:msg -> bug 55464 mh 3510 mfmediaengine:mfmediaengine -> fixed bug 55422 (MR!3351 interference) 2023-08-04 --- week 2023-07-31 ro unknown secur32:negotiate -> fixed bug 55627 ro unknown secur32:ntlm -> fixed bug 55627 2023-07-28 -- week 2023-07-24 2023-07-27 mf 3389 build:mac -> fixed bug 55366, -lpcsclite not found (configure bug) mm 3401 comctl32:listbox -> bug 55394 mg 3398 gdiplus:graphicspath -> fixed bug 55351 2023-07-21 -- week 2023-07-17 2023-07-20 r? unknown ddraw:ddraw7 -> bug 55349 2023-07-19 r? unknown ntdll:exception -> bug 55480 2023-07-18 r? unknown winmm:mci -> bug 55605 2023-07-17 mo 3336 advapi32:registry -> bug 55288 (32-after-64) mo 3336 ntdll:reg -> bug 55288 (32-after-64) 2023-07-14 --- week 2023-07-10 r? unknown user32:msg -> bug 55786 r? unknown d3dx10_39:d3dx10 -> fixed bug 55223 r? unknown d3dx10_42:d3dx10 -> fixed bug 55223 2023-07-13 cf 5799e89f15b5 itss:protocol -> bug 55460 (deb12) 2023-07-12 mb 3050 mshtml:misc -> fixed bug 55244 (bad for mshtml:dom crash) 2023-07-11 r? unknown psapi:psapi_main -> bug 55416 c. 7e521b4db353 ntdll:info -> bug 55269 r? unknown vbscript:createobj -> bug 55330 2023-07-10 cw 5799e89f15b5 amstream:amstream -> fixed bug 55283 (deb12) cw 5799e89f15b5 gdi32:font -> bug 55267 (deb12) cw 5799e89f15b5 kernel32:loader -> bug 55228 (deb12) cw 5799e89f15b5 kernel32:loader -> bug 55229 (deb12) cw 5799e89f15b5 kernel32:loader -> bug 55266 (deb12) cw 5799e89f15b5 kernel32:process -> bug 55226 (deb12) cw 5799e89f15b5 kernel32:thread -> fixed bug 55225 (deb12) cw 5799e89f15b5 ntdll:om -> fixed bug 55245 (deb12) cw 5799e89f15b5 ntdll:om -> fixed bug 55246 (deb12) cw 5799e89f15b5 ntdll:wow64 -> fixed bug 55273 (deb12) cw 5799e89f15b5 oleaut32:usrmarshal -> fixed bug 55237 (deb12) cw 5799e89f15b5 scrrun:dictionary -> fixed bug 55236 (deb12) cw 5799e89f15b5 winmm:capture -> fixed bug 55240 (deb12) cw 5799e89f15b5 winmm:capture -> fixed bug 55241 (deb12) cw 5799e89f15b5 winmm:wave -> fixed bug 55240 (deb12) cw 5799e89f15b5 winmm:wave -> fixed bug 55241 (deb12) cw 5799e89f15b5 wtsapi32:wtsapi -> fixed bug 55227 (deb12) mo 3274 gdi32:dc -> bug 55291 (fg-deb64) 2023-07-07 -- week 2023-07-03 2023-07-05 mb 3049 user32:text -> bug 55215 2023-07-04 c. 36963b6d27ab oleaut32:vartest -> fixed bug 55194 2023-06-30 -- week 2023-06-26 2023-06-28 r? unknown d3dx9_36:effect -> bug 55478 2023-06-27 me 3122 ntdll:file -> fixed bug 55331 (1607 & 1709) 2023-06-23 --- week 2023-06-19 mi 3129 dbghelp:path -> fixed bug 55128 (entry point) 2023-06-22 mf 3141 secur32:schannel -> bug 55114 failure seemed systematic though r? unknown user32:msg -> bug 55374 r? unknown user32:msg -> fixed bug 55354 2023-06-21 c. 2ef4cc48fbce dbghelp:dbghelp -> bug 55109 c. 2ef4cc48fbce kernel32:debugger -> bug 55120 r? unknown mmdevapi:render -> bug 55727 r? unknown ntdll:exception -> bug 55111 2023-06-19 mt 3080 xaudio2_8:xaudio2 -> bug 55108 (xaudio2_7) 2023-06-16 --- week 2023-06-12 r? unknown psapi:psapi_main -> bug 55429 2023-06-15 r? user32:win -> bug 55778 2023-06-14 r? unknown d3dx10_34:d3dx10 -> fixed bug 55223 r? unknown d3dx10_35:d3dx10 -> fixed bug 55223 2023-06-13 mf 3055 quartz:vmr7 -> bug 55110 mf 3055 quartz:vmr9 -> bug 55110 2023-06-12 c. cc2cfb9b792b dbghelp:dbghelp -> bug 55058 r? unknown user32:msg -> bug 55465 r? unknown user32:msg -> bug 55463 2023-06-09 --- week 2023-06-05 r? unknown ntdll:threadpool -> bug 55493 2023-06-07 mt 2918 xaudio2_8:xaudio2 -> bug 55016 (xaudio2_7) 2023-06-05 r? unknown user32:cursoricon -> bug 55424 2023-06-02 --- week 2023-05-29 c. b337c5b1a175 kernel32:comm -> bug 55014 mb 2950 user32:monitor -> bug 55013 2023-06-01 r? unknown shell32:systray -> bug 55017 2023-05-31 mf 2901 mf:transform -> bug 55638 2023-05-26 --- week 2023-05-22 2023-05-19 --- week 2023-05-15 r? unknown user32:msg -> bug 55066 2023-05-15 r? unknown win32u:win32u -> bug 54975 2023-05-12 --- week 2023-05-08 2023-05-05 --- week 2023-05-01 2023-05-04 r? unknown imm32:imm32 -> bug 55491 2023-05-01 me 2678 user32:monitor -> bug 55065 (debiant) 2023-04-28 --- week 2023-04-24 2023-04-21 --- week 2023-04-17 2023-04-17 me 2659 imm32:imm32 -> bug 54864 (Korean locale) 2023-04-14 --- week 2023-04-10 2023-04-13 mb 2647 dnsapi:query -> bug 54847 mb 2647 dnsapi:query -> bug 54848 2023-04-07 --- week 2023-04-03 2023-03-31 --- week 2023-03-27 2023-03-29 mw 2531 imm32:imm32 -> bug 54760 2023-03-24 --- week 2023-03-20 2023-03-22 mo 2445 advapi32:registry -> bug 54742 (32-after-64) mo 2445 msi:action -> bug 54738 (64-bit) mo 2445 ole32:compobj -> bug 54751 (only in 64-bit full suite) mo 2445 oleaut32:typelib -> bug 54739 (64-bit) 2023-03-17 --- week 2023-03-13 r? unknown adsldp:ldap -> bug 54707 (ldap server outage) 2023-03-16 mb 2421 dinput:device8 -> bug 54713 me 2419 imm32:imm32 -> bug 54710 (Korean locale) me 2419 imm32:imm32 -> bug 54711 (Korean locale) 2023-03-15 mo 1445 reg.exe:copy -> bug 54691 (64-bit) mo 1445 reg.exe:delete -> bug 54691 (64-bit) mo 1445 reg.exe:export -> bug 54691 (64-bit) mo 1445 reg.exe:import -> bug 54691 (64-bit) mo 1445 reg.exe:query -> bug 54691 (64-bit) mo 1445 reg.exe:add -> bug 54691 (32-after-64) 2023-03-14 c. c44e58c88bd1 ntdll:info -> bug 54688 2023-03-13 r? unknown shlwapi:ordinal -> bug 54683 2023-03-10 --- week 2023-03-06 2023-03-09 me 2354 imm32:imm32 -> bug 54669 (Korean & Mixed locales) mb 2361 kernel32:loader -> bug 54657 2023-03-08 mf 2347 dbghelp:dbghelp -> bug 54717 (rare) mb 2252 psapi:psapi_main -> bug 55010 mw 2234 user32:input -> bug 55327 2023-03-06 2023-03-03 --- week 2023-02-27 r? unknown dinput:device8 -> bug 54774 2023-03-02 r? unknown shell32:progman_dde -> bug 54758 mf 2313 shlwapi:ordinal -> bug 54683 (summertime-specific) 2023-02-28 r? unknown comdlg32:filedlg -> bug 55153 2023-02-27 2023-02-24 --- week 2023-02-20 2023-02-23 r? unknown d3d11:d3d11 -> bug 54757 2023-02-22 mo 2250 winscard:winscard -> bug 54661 (missing build dep) 2023-02-17 --- week 2023-02-13 2023-02-10 --- week 2023-02-06 mb 2154 user32:msg -> fixed bug 54477 2023-02-09 r? unknown httpapi:httpapi -> bug 55127 2023-02-03 --- week 2023-01-30 2023-01-27 --- week 2023-01-23 2023-01-20 --- week 2023-01-16 2023-01-13 --- week 2023-01-09 2023-01-06 --- week 2023-01-02 2022-12-23 --- week 2022-12-19 2022-12-22 mf 1874 user32:msg -> fixed, no bug 2022-12-16 --- week 2022-12-12 2022-12-09 --- week 2022-12-02 2022-12-07 r? unknown explorerframe:nstc -> bug 54791 mb 1732 taskschd:scheduler -> fixed bug 54105 2022-12-02 --- week 2022-11-28 2022-11-26 --- week 2022-11-21 2022-11-22 mb 1495 vbscript:run -> fixed since 2022-11-18 --- week 2022-11-14 2022-11-17 mb 1399 all:tests -> bug 53963 mb 1398 ntoskrnl.exe:ntoskrnl -> bug 53976 2022-11-11 -- week 2022-11-07