[Bug 43880] New: Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

Bug ID: 43880
           Summary: Testbot will not submit results (too many errors)
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86-64
                OS: Windows
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: [e-mail]
          Reporter: [e-mail]

"59 tests failed, there's probably something broken with your setup.
You need to address this before submitting results."

52 errors for the 64 bit test, and 59 errors for the 32 bit test.
I'm running this on real hardware, Vista, as administrator.  Please update the
testing software so that I can submit the results, even with a high number of
errors.

And while you're at it, I would change the comma after "failed" to a colon.

I looked for similar bug reports but found nothing expressly dealing with this.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #1 from Alexandre Julliard  ---
This is on purpose, to avoid polluting the test results with a lot of spurious
failures. Which tests are failing for you?

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #2 from [e-mail] ---
(In reply to Alexandre Julliard from comment #1)
> This is on purpose, to avoid polluting the test results with a lot of
> spurious failures. Which tests are failing for you?

How do I tell which ones are failing?  I'm not sure where the report that gets
sent to you all is saved on my computer while the tests are running.  The
little test window cuts off where the file is saved to, cuts off other text,
doesn't currently give the option to view the report before sending it.

I've run wine tests before and usually have less than 40 errors.  If you look
on the winetest results page, you will see under Vista where there is a high
number.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #3 from [e-mail] ---
I believe I have found out why I have had so many bugs.

It's because if I run the winetest in Windows, and then turn on the Windows
lock screen, I get something like 20 more errors that I wouldn't have otherwise
gotten if I had kept the lock screen off.  Should I file a new bug for this?  I
hope somebody reads this.

Anyways, with the lock screen on, I get something like 57-61 bugs, around
there, but if I have it off, something like 39-44.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #4 from [e-mail] ---
Also, with the lock screen on, then the test bot comes up with a error saying
that it can't delete a temporary folder.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

SWAT  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #5 from SWAT  ---
I'm getting 60 errors when running the testsuite on Windows 10 on physical
hardware and am also missing information on where to find the results (and what
to do with them). Basically I'm having the same or extremely similar issues as
the submitter. How should I proceed to get this fixed (what can I do to help)?

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #6 from François Gouget  ---
(In reply to parkerjbarker from comment #3)
[...]
> It's because if I run the winetest in Windows, and then turn on the Windows
> lock screen, I get something like 20 more errors that I wouldn't have
> otherwise gotten if I had kept the lock screen off.

You shouldn't lock the screen when running the Wine tests. It is however
possible that the screen locks up automatically. Maybe the WineTest executable
should do something to prevent that from happening if that's possible. But the
safest would the to disable screen locking (and maybe the screen saver too)
when running the tests.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #7 from François Gouget  ---
(In reply to parkerjbarker from comment #2)
[...]
> How do I tell which ones are failing?  I'm not sure where the report that
> gets sent to you all is saved on my computer while the tests are running.

You can get the report that the website rejected by going to the URL below in
the following days:
https://test.winehq.org/data/errors.html

However the best approach is to run the WineTest from the console and to tell
it where to save the report:

winetest.exe -c -t YOUR-TAG-HERE -m YOUR-EMAIL-HERE -o winetest.report \
       2>winetest.log

The report will be in winetest.report. Look for the "tests executed" lines that
have a non-zero number of failures. You can also look for the places where the
number of failures change on the "Running tests" lines in winetest.log.

See also winetest.exe --help for further details.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #8 from [e-mail] ---
(In reply to Alexandre Julliard from comment #1)
> This is on purpose, to avoid polluting the test results with a lot of
> spurious failures. Which tests are failing for you?

I figure now that it is code freeze, it might be a good idea to increase the
limit for submitting bugs in Win7 past 50, like maybe 70 would be a good
number, temporarily.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

--- Comment #9 from François Gouget  ---
We normally increase the limit when there is a new version of Windows: those
tend to have a higher number of failures because the API behavior changed. But
that's not the case with a Wine freeze so I don't think there's a reason to
increase the limit now.

Could run WineTest from the command line, post the log somewhere, pick a test
that does not have failures on test.winehq.org and ask on wine-devel for help
to figure out why it fails for you? Do mention there if there anything specific
about your configuration: locale, specific fonts, maybe some unusual graphics
card, high DPI screen, lack of a sound card, etc.

[Bug 43880] Testbot will not submit results (too many errors)

https://bugs.winehq.org/show_bug.cgi?id=43880

jim  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

[Bug 43880] Testbot will not submit results (too many errors)

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=43880

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |NOTOURBUG

--- Comment #10 from François Gouget  ---
As discussed this does not look like a TestBot bug: figuring out why this
system has so many failures is best done through the devel mailing list or IRC.
So I am closing this bug.