[Bug 35522] New: Send a test to testbot using 1 page instead of 3

http://bugs.winehq.org/show_bug.cgi?id=35522

Bug ID: 35522
           Summary: Send a test to testbot using 1 page instead of 3
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: [e-mail]
          Reporter: [e-mail]
    Classification: Unclassified

Currently it is:
1. Select a file to send
2. Select the VMs
3. Select additional flags (command line arguments, whether to run 64-bit
tests, debug level, report successful tests)

One option would be to merge all 3 pages into 1. Another possibility is to
add option "Use defaults" and activate it by default to skip 2 and 3 pages
using default settings (which in most cases nobody cares about).

[Bug 35522] Send a test to testbot using 1 page instead of 3

http://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #1 from Bruno Jesus  ---
That would be useful for cross-compiled exe tests. But for patches I don't
think it's possible because it analyzes the uploaded patch file to auto-fill
some parameters in the third screen.

I like the second option =)

[Bug 35522] Send a test to testbot using 1 page instead of 3

http://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #2 from Dmitry Timoshkov  ---
Another option would be to add buttons "Select the VMs" and "Select
additional run flags" to 1st screen and use defaults if the sender
doesn't care about them.

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

François Gouget  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
                 CC|                            |[e-mail]
             Status|NEW                         |RESOLVED

--- Comment #3 from François Gouget  ---
This is done.

commit 0ad777fa1db816a0f74663c94d9fa969218ac859
Author: Francois Gouget 
Date:   Wed Oct 2 06:56:23 2019 +0200

testbot/Submit: Add a single page mode.

Signed-off-by: Francois Gouget 
    Signed-off-by: Alexandre Julliard

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #4 from Dmitry Timoshkov  ---
(In reply to François Gouget from comment #3)
> This is done.
>
> commit 0ad777fa1db816a0f74663c94d9fa969218ac859
> Author: Francois Gouget 
> Date:   Wed Oct 2 06:56:23 2019 +0200
> testbot/Submit: Add a single page mode.
>
> Signed-off-by: Francois Gouget <[e-mail]>
> Signed-off-by: Alexandre Julliard <[e-mail]>

Thanks François!

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

Dmitry Timoshkov  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|FIXED                       |---
             Status|RESOLVED                    |REOPENED

--- Comment #5 from Dmitry Timoshkov  ---
I tested this today and while new page supposedly works for precompiled
test exes it's still is not implemented for .diffs, which is the primary
target for Wine tests.

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #6 from François Gouget  ---
It works for me, see for instance job 35522:
https://bugs.winehq.org/show_bug.cgi?id=35522

It does require the submitter to pick the executable test name and parameters
since the TestBot does not get a chance to analyze the path and provide
defaults before the job submission.

Now you said this is not implemented. I assume you got an error message of some
sort.
What was the error message?
Can you provide a test case to reproduce this?
 - Patch
 - Selected VMs
 - Remarks field
 - Test executable field
 - Command line arguments field

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #7 from Dmitry Timoshkov  ---
(In reply to François Gouget from comment #6)
> It works for me, see for instance job 35522:
> https://bugs.winehq.org/show_bug.cgi?id=35522
>
> It does require the submitter to pick the executable test name and
> parameters since the TestBot does not get a chance to analyze the path and
> provide defaults before the job submission.

I don't follow your statement above, everything testbot needs to know is
contained in the .diff file.
> Now you said this is not implemented. I assume you got an error message of
> some sort.
> What was the error message?
> Can you provide a test case to reproduce this?
> - Patch
> - Selected VMs
> - Remarks field
> - Test executable field
> - Command line arguments field

Steps to reproduce:
1. create any NOP test change
2. create patch.diff
3. goto https://testbot.winehq.org/Submit.pl
4. select "Single Page"
5. select "Choose File"
6. press "Submit"

The error message is "Invalid test executable filename".

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #8 from François Gouget  ---
That's because you left the "Test Executable" field empty, which is not a valid
test executable name.

Also notice that patches may modify more than one test module in which case the
TestBot has no way of knowing which one you intended to test (jobs submitted
through the website are still limited to running a single test unit). But in
the simple case it could indeed provide a default value if that field is left
blank.

Notice however that for the "Command line arguments" field it cannot know if
you really intended to pass no parameter to the test executable or whether a
default value should be provided (and there too the value to use could be
ambiguous).

A way to lift this ambiguity would be to have a box that must be checked to
provide custom command line arguments. But that would really only be useful for
the Single Page mode since in the normal case a default is provided.

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #9 from Dmitry Timoshkov  ---
(In reply to François Gouget from comment #8)
> That's because you left the "Test Executable" field empty, which is not a
> valid test executable name.

It's not required when submitting a .diff in multiple steps, I don't see
why this should differ in a single page UI.
> Also notice that patches may modify more than one test module in which case
> the TestBot has no way of knowing which one you intended to test (jobs
> submitted through the website are still limited to running a single test
> unit). But in the simple case it could indeed provide a default value if
> that field is left blank.

Once again, this isn't different from a multi-page UI.
> Notice however that for the "Command line arguments" field it cannot know if
> you really intended to pass no parameter to the test executable or whether a
> default value should be provided (and there too the value to use could be
> ambiguous).
>
> A way to lift this ambiguity would be to have a box that must be checked to
> provide custom command line arguments. But that would really only be useful
> for the Single Page mode since in the normal case a default is provided.

Again, I don't see why you are trying to make single page UI to require
anything more that multi-page UI does not.

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #10 from François Gouget  ---
(In reply to Dmitry Timoshkov from comment #9)
[...]
> > But in the simple case it could indeed provide a default value if
> > that field is left blank.
>
> Once again, this isn't different from a multi-page UI.

The difference is that in the multi-page UI the TestBot knows what's in the dif
before showing you the Test Executable field, whereas in the single page case
it only has access to the diff not only after putting up that field, but also
after you click on the Submit button.

I think you can work out the UI implications from there.

[Bug 35522] Send a test to testbot using 1 page instead of 3

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=35522

--- Comment #11 from Dmitry Timoshkov  ---
(In reply to François Gouget from comment #10)
> (In reply to Dmitry Timoshkov from comment #9)
> [...]
> > > But in the simple case it could indeed provide a default value if
> > > that field is left blank.
> >
> > Once again, this isn't different from a multi-page UI.
>
> The difference is that in the multi-page UI the TestBot knows what's in the
> dif before showing you the Test Executable field, whereas in the single page
> case it only has access to the diff not only after putting up that field,
> but also after you click on the Submit button.

What kind of a limitation prevents tesbot to figure out anything about
the submitted test file at any step? Why it needs to know the executable
name at all?
> I think you can work out the UI implications from there.

The whole point of a single page UI is to point out tesbot a file (.diff or
.exe) and be done with it.