[PATCH 0/1] MR990: comctl32/tests: Mark tests failing randomly on Windows as flaky.

[PATCH 1/1] comctl32/tests: Mark the monthcal tests failing on Wednesdays as flaky.

[Francois Gouget]

From: Francois Gouget

Wine-Bug: https://bugs.winehq.org//show_bug.cgi?id=50097
---
 dlls/comctl32/tests/monthcal.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/dlls/comctl32/tests/monthcal.c b/dlls/comctl32/tests/monthcal.c
index fcafb790302..0abbcd9764d 100644
--- a/dlls/comctl32/tests/monthcal.c
+++ b/dlls/comctl32/tests/monthcal.c
@@ -2051,6 +2051,7 @@ static void test_sel_notify(void)
         got_MCN_SELECT = got_MCN_SELCHANGE = FALSE;
         SendMessageA(hwnd, WM_LBUTTONDOWN, 0, MAKELPARAM(mchit.pt.x, mchit.pt.y));
         if (styles[i].val & MCS_MULTISELECT)
+            flaky_if(i == 3)
             ok(got_MCN_SELCHANGE, "%d: MCN_SELCHANGE should be sent\n", i);
         else
             ok(!got_MCN_SELCHANGE, "%d: MCN_SELCHANGE should not be sent\n", i);
@@ -2060,6 +2061,7 @@ static void test_sel_notify(void)
         if (styles[i].val & MCS_MULTISELECT)
             ok(!got_MCN_SELCHANGE, "%d: MCN_SELCHANGE should not be sent\n", i);
         else
+            flaky_if(i == 2)
             ok(got_MCN_SELCHANGE, "%d: MCN_SELCHANGE should be sent\n", i);
         ok(got_MCN_SELECT, "%d: MCN_SELECT should be sent\n", i);
         DestroyWindow(hwnd);

[Nikolay Sivov (＠nsivov)]

Is "flaky" intended to be used on Windows as well? Before we usually considered it a broken test that needs fixing, if it doesn't work on Windows.

[Francois Gouget (＠fgouget)]

broken() would be used if a given Windows version was systematically returning a different value from what we are expecting. But most of the time Windows returns the value we expect. So from that point of view flaky() would be appropriate.

However in fact the test does not fail "randomly". instead it systematically fails on Wednesdays. So flagging it as "flaky" may going beyond the intended scope for flaky. But then that scope was never really defined. So this is the perfect opportunity for defining those boundaries...

Your other options are:
* Fixing the test.
* Putting it in an if (0).
* Removing it.

[PATCH 0/1] MR990: comctl32/tests: Mark tests failing randomly on Windows as flaky. - closed

[Zhiyi Zhang (＠zhiyi)]

This merge request was closed by Zhiyi Zhang.

[Zhiyi Zhang (＠zhiyi)]

The bug should be fixed by 190fbc750 now.