GitLab CI's Most Wanted list

[Francois Gouget]

Processing the failures-mr.txt file for the past month (see previous 
email) points to the following test units as plagging the most merge 
requests:

100 msxml3:domdoc         (55720)
     87 amstream:amstream     (55725)
     81 quartz:filtergraph    (55758)
     44 build:mac
     26 d2d1:d2d1             (55765)
     19 winmm:mci             (55605)
     15 user32:msg            (55786, mostly)
     13 user32:win            (55766)
     12 psapi:psapi_main      (55744, fixed on day 3)
     11 wscript.exe:run       (55746, fixed on day 3)
     11 tasklist.exe:tasklist (55745, fixed on day 3)

Good hunting!

[Eric Pouech]

Le 20/10/2023 à 10:49, Francois Gouget a écrit :
> Processing the failures-mr.txt file for the past month (see previous
> email) points to the following test units as plagging the most merge
> requests:
>  100 msxml3:domdoc         (55720)
>   87 amstream:amstream     (55725)
>   81 quartz:filtergraph    (55758)
>   44 build:mac
>   26 d2d1:d2d1             (55765)
>   19 winmm:mci             (55605)
>   15 user32:msg            (55786, mostly)
>   13 user32:win            (55766)
>   12 psapi:psapi_main      (55744, fixed on day 3)
>   11 wscript.exe:run       (55746, fixed on day 3)
>   11 tasklist.exe:tasklist (55745, fixed on day 3)
>
> Good hunting!

I've tried to repro a couple of them locally, recreating the docker 
image with the docker script... to no avail

quite a few failures (amstream, mci) (didn't look at all of them) happen 
with a llvm error + wine assertion fail + syscall backtrace

from some perusing of this kind of failure:

- these failures are quite recurrent (almost all the times) on test-linux32

- found such a failure up to the 15 of July, but a good clean run the 8 
july (3272)

- what happened in between? taking apart French day, docker image has 
been recreated with bookworm on July 13th

perhaps a good starting point would be ot update the docker image

HTH

[Francois Gouget]

On Fri, 20 Oct 2023, Eric Pouech wrote:
[...]
> I've tried to repro a couple of them locally, recreating the docker image with
> the docker script... to no avail

I have a question on that subject: how do you run the Wine tests in your 
local docker image?

The reason I ask is that reproducing GitLab CI's container and using it 
for builds is pretty easy. But running the tests in it requires starting 
X and the commands (in test.yml) for that are not directly usable 
outside the GitLab environment.

So do you copy/paste all the xinitrc, startx, wine-mono & co stuff to a 
script, append the ./wine command you want to run at the end and do 
something like "docker run gitlab-debian12 ./myscript"?

That also assumes mounting the current directory somewhere in the 
container but having a script that does echo >~/.xinitrc sounds like a 
disaster waiting to happen if it's accidentally run in the wrong 
environment.

Or did you add a RUN directive in the Dockerfile to start X and then 
'docker exec' your test in it?

I feel like it would be nice if tools/gitlab contained a script or 
something to make this whole process easier. Developers have actually 
asked me [1] "how can I replicate the GitLab CI environment" a few times 
already.

[1] Why ask me? Am I the GitLab CI maintainer now?

[Eric Pouech]

Le 20/10/2023 à 14:37, Francois Gouget a écrit :
> On Fri, 20 Oct 2023, Eric Pouech wrote:
> [...]
> > I've tried to repro a couple of them locally, recreating the docker image with
> > the docker script... to no avail
>
> I have a question on that subject: how do you run the Wine tests in your
> local docker image?
>
> The reason I ask is that reproducing GitLab CI's container and using it
> for builds is pretty easy. But running the tests in it requires starting
> X and the commands (in test.yml) for that are not directly usable
> outside the GitLab environment.
>
> So do you copy/paste all the xinitrc, startx, wine-mono & co stuff to a
> script, append the ./wine command you want to run at the end and do
> something like "docker run gitlab-debian12 ./myscript"?

yes something along these lines [2]

- mounting build directory

- mounting script directory (with some of the scripts being a direct 
copy of tools/gitlab, others being a more or less equivalent 
transformations of the .yml files into bash scripts)

- actually split in stages (create image + install wine & deps, compile 
wine (optionnaly with SHA1+fetch+rebase), run winetest (with options for 
the tests to be run)

pitfalls:

- not using the exact same scripts as the gitlab ones...

- can't get official docker image from gitlab's repo (so a new one is 
built from docker repo for debian image)

- not running on the exact same OS as the HW used for gitlab's pipeline 
(which I don't even know the version... so didn't go to the point of 
spinning a debian vm to run docker on it...)

- is gitlab applying some other parameters when running image (memory 
limit, swap limit???). idk

my initial goal wasn't to be able to run it regularly, but mainling to 
try to replicate pipeline failures, which I couldn't get to that point

discrepancies could come from pitfalls above
> [1] Why ask me? Am I the GitLab CI maintainer now?

[2] What are the implications of answering?

[Zeb Figura]

On Friday, 20 October 2023 03:49:50 CDT Francois Gouget wrote:
> Processing the failures-mr.txt file for the past month (see previous 
> email) points to the following test units as plagging the most merge 
> requests:
> 100 msxml3:domdoc         (55720)
>  87 amstream:amstream     (55725)
>  81 quartz:filtergraph    (55758)
>  44 build:mac
>  26 d2d1:d2d1             (55765)
>  19 winmm:mci             (55605)
>  15 user32:msg            (55786, mostly)
>  13 user32:win            (55766)
>  12 psapi:psapi_main      (55744, fixed on day 3)
>  11 wscript.exe:run       (55746, fixed on day 3)
>  11 tasklist.exe:tasklist (55745, fixed on day 3)
>
> Good hunting!

I'll look into the DirectShow ones (quartz, amstream, mci).