[PATCH v6 0/2] MR271: Introduce a GitLab pipeline for vkd3d
With this pipeline vkd3d automatically gets built on Linux (in an image based on Debian unstable), both in 32 and 64 bit mode. Both builds are tested with radv and llvmpipe. A number of caveats apply, though: * A number of tests currently fail in llvmpipe, so the llvmpipe jobs are marked as allowed to fail. Ideally we'll eventually fix our bugs and mark the llvmpipe ones in the tests, so that the CI tests completely pass and possible problems in the Vulkan driver are recorded at a better granularity (this is the reason why GitLab says that the pipeline is passed with warnings: the warnings are that there are jobs that failed, even if they were allowed to fail). * ~~The runners provided by the GitLab instance don't have a GPU available, so I configured my own computer (equipped with an AMD Radeon RX 5700 RX) to provide a runner with access to the GPU. This setup is not currently satisfying: for me, because I use that computer for other things and I don't like having random code submitted to it (it is theoretically sandboxed, but sandboxes are not always bullet-proof, especially if they have access to a GPU); for the users, because my computer might be unavailable at any time. I'll work on a better solution. For the time being I intend the runner to only accept jobs from the master branch; once a better solution is implemented I'd like to run the pipeline for MRs too.~~ **Fixed**, now there is a shared worker with an AMD GPU available. * While the `Dockerfile` and related assets do not necessarily need to be available in this repository, given that the CI accesses the binary image from the Docker hub anyway, I think it's still valuable to have them, so others can improve them (and for sure improvement opportunities are nowhere near missing). However, other ways to make them available can be found, if for some reason it is not liked to have them in this repository (they are not pretty!). * ~~One of the reason they are not pretty is that I have a custom hack to compile `widl` from the Wine sources without compiling (or installing from the distribution) the whole of Wine, in the interest of keeping the Docker image small (well, small-ish: Vulkan drivers, compilers and X libraries are not small anyway).~~ **A better solution was implemented** * ~~Again on the subject of the Docker image, I am currently putting the binary image in my own namespace on the Docker hub. Using the GitLab container registry in the namespace of the Wine project would probably be better, so that I am not a bottleneck in the future.~~ **Done** * ~~Even if we discount all the points above, this MR cannot be merged yet, because my runner is currently configured for my namespace only. I guess I need the intervention of a GitLab admin to fix that. However, I think there's already material enough for valuable feedback.~~ **Fixed too** -- v6: ci: Introduce a CI pipeline for GitLab. https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271
From: Giovanni Mascellani <gmascellani(a)codeweavers.com> Commit b7402ddbbecdfaa81daa657fbb5d37661f401434 from wine is required, which was first released with 3.21, not 3.20. --- README | 4 ++-- configure.ac | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README b/README index 465d5f915..4989f9e9a 100644 --- a/README +++ b/README @@ -15,8 +15,8 @@ Vkd3d generates some of its headers from IDL files. If you are using the release tarballs, then these headers are pre-generated and are included. If you are building from git, then they will be generated at build-time using widl. By default, vkd3d will use the widl found in `PATH'. If widl is not -available or is not recent (>= 3.20), then you can build Wine with `make -tools/widl' to avoid building all of Wine. You can then point vkd3d's +available or is not recent (>= 3.21), then you can build Wine with `make +tools/widl/widl' to avoid building all of Wine. You can then point vkd3d's configure at that widl binary with `WIDL="/path/to/widl"'. For release builds, you may want to define NDEBUG. If you do not need debug log diff --git a/configure.ac b/configure.ac index b58c2f4d2..213ab22ee 100644 --- a/configure.ac +++ b/configure.ac @@ -27,7 +27,7 @@ AC_PROG_CC AM_PROG_CC_C_O AC_PROG_SED AC_PROG_MKDIR_P -VKD3D_PROG_WIDL(3, 20) +VKD3D_PROG_WIDL(3, 21) AS_IF([test "x$WIDL" = "xno"], [AC_MSG_WARN([widl is required to build header files.])]) AC_CHECK_PROGS([FLEX], [flex], [none]) -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271
From: Giovanni Mascellani <gmascellani(a)codeweavers.com> --- .gitlab-ci.yml | 7 +++++++ gitlab/build-linux | 19 +++++++++++++++++++ gitlab/build.yml | 29 +++++++++++++++++++++++++++++ gitlab/image.docker | 39 +++++++++++++++++++++++++++++++++++++++ gitlab/image.yml | 20 ++++++++++++++++++++ 5 files changed, 114 insertions(+) create mode 100644 .gitlab-ci.yml create mode 100755 gitlab/build-linux create mode 100644 gitlab/build.yml create mode 100644 gitlab/image.docker create mode 100644 gitlab/image.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 000000000..8cc59d764 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,7 @@ +stages: + - image + - build + +include: + - local: "/gitlab/image.yml" + - local: "/gitlab/build.yml" diff --git a/gitlab/build-linux b/gitlab/build-linux new file mode 100755 index 000000000..1b7a7a5b9 --- /dev/null +++ b/gitlab/build-linux @@ -0,0 +1,19 @@ +#!/bin/bash + +echo "Building $(git log -1)" +echo "---" + +COMMIT=$(git rev-parse --short HEAD) + +set -Eeuxo pipefail + +./autogen.sh +rm -fr build +mkdir build +cd build +../configure --enable-demos && make -j$(nproc) && make -j$(nproc) crosstest || touch ../pipeline_failed + +mkdir -p ../artifacts/$COMMIT +rsync -Rr doc/* tests/*.exe ../artifacts/$COMMIT + +git reset --hard diff --git a/gitlab/build.yml b/gitlab/build.yml new file mode 100644 index 000000000..49e69d575 --- /dev/null +++ b/gitlab/build.yml @@ -0,0 +1,29 @@ +.build: + stage: build + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' + image: $CI_REGISTRY/wine/vkd3d:debian-bookworm + interruptible: true + needs: + - job: build-image + optional: true + dependencies: [] + script: + - git config --global --add safe.directory $CI_PROJECT_DIR + - git clean -fdx + - git reset --hard + - rm -fr .git/rebase-merge + - mkdir artifacts + - cat /proc/cpuinfo > artifacts/cpuinfo.txt + - cat /proc/meminfo > artifacts/meminfo.txt + - vulkaninfo > artifacts/vulkaninfo.txt + - git rebase $CI_MERGE_REQUEST_DIFF_BASE_SHA --exec ./gitlab/build-linux + - if [ -f pipeline_failed ] ; then exit 1 ; fi + artifacts: + when: always + expire_in: 1 day + paths: + - artifacts + +build: + extends: .build diff --git a/gitlab/image.docker b/gitlab/image.docker new file mode 100644 index 000000000..5f26940d2 --- /dev/null +++ b/gitlab/image.docker @@ -0,0 +1,39 @@ +WORKDIR /tmp + +RUN export DEBIAN_FRONTEND=noninteractive; \ + echo 'path-exclude=/usr/share/doc/*' > /etc/dpkg/dpkg.cfg.d/99-exclude-cruft && \ + echo 'path-exclude=/usr/share/locale/*' >> /etc/dpkg/dpkg.cfg.d/99-exclude-cruft && \ + echo 'path-exclude=/usr/share/man/*' >> /etc/dpkg/dpkg.cfg.d/99-exclude-cruft && \ + echo 'APT::Install-Recommends "false";' > /etc/apt/apt.conf && \ + echo '#!/bin/sh' > /usr/sbin/policy-rc.d && \ + echo 'exit 101' >> /usr/sbin/policy-rc.d && \ + chmod +x /usr/sbin/policy-rc.d && \ + dpkg --add-architecture i386 && \ + apt-get update && \ + apt-get install -y eatmydata && \ + eatmydata apt-get dist-upgrade -y && \ + eatmydata apt-get install -y build-essential pkg-config gcc-multilib gcc-mingw-w64 \ + autoconf automake libtool flex bison \ + git ca-certificates rsync \ + doxygen doxygen-latex graphviz \ + mesa-vulkan-drivers mesa-vulkan-drivers:i386 \ + vulkan-tools spirv-headers \ + libvulkan-dev libvulkan-dev:i386 \ + libncurses-dev libncurses-dev:i386 \ + libxcb1-dev libxcb1-dev:i386 \ + libxcb-util-dev libxcb-util-dev:i386 \ + libxcb-icccm4-dev libxcb-icccm4-dev:i386 \ + libxcb-keysyms1-dev libxcb-keysyms1-dev:i386 && \ + git clone --depth 1 --branch wine-3.21 https://gitlab.winehq.org/wine/wine.git && \ + cd wine && \ + mkdir build && \ + cd build && \ + ../configure --without-x --without-freetype && \ + make tools/widl/widl && \ + cp tools/widl/widl /usr/local/bin && \ + cd ../.. && \ + rm -rf wine && \ + apt-get clean && \ + useradd -m gitlab + +USER gitlab \ No newline at end of file diff --git a/gitlab/image.yml b/gitlab/image.yml new file mode 100644 index 000000000..e1be68b0e --- /dev/null +++ b/gitlab/image.yml @@ -0,0 +1,20 @@ +build-image: + stage: image + rules: + - if: $CI_PIPELINE_SOURCE == 'push' && $CI_PROJECT_PATH == "wine/vkd3d" && $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + changes: + - .gitlab-ci.yml + - gitlab/image.yml + - gitlab/image.docker + image: + name: gcr.io/kaniko-project/executor:debug + entrypoint: [""] + variables: + IMAGE_SOURCE: "gitlab.winehq.org:443/wine/dependency_proxy/containers/debian:bookworm" + IMAGE_LOCAL: "$CI_REGISTRY_IMAGE:debian-bookworm" + DOCKER_FILE: "$CI_PROJECT_DIR/gitlab/image.docker" + script: + - mkdir -p /kaniko/.docker + - echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(printf "%s:%s" "$CI_REGISTRY_USER" "$CI_REGISTRY_PASSWORD" | base64 | tr -d '\n')\"},\"$CI_DEPENDENCY_PROXY_SERVER\":{\"auth\":\"$(printf "%s:%s" "$CI_DEPENDENCY_PROXY_USER" "$CI_DEPENDENCY_PROXY_PASSWORD" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json + - sed -i "1iFROM $IMAGE_SOURCE" "$DOCKER_FILE" + - /kaniko/executor --context "$CI_PROJECT_DIR" --dockerfile "$DOCKER_FILE" --destination "$IMAGE_LOCAL" -- GitLab https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271
Alexandre asked to begin with a simpler pipeline and then add more features once it is seen that it is behaving well. -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271#note_39748
Alexandre asked to begin with a simpler pipeline and then add more features once it is seen that it is behaving well.
Thanks for doing this, it looks good to me. -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271#note_43581
```diff +USER gitlab \ No newline at end of file ```
Could we get a newline? :) -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271#note_43662
This merge request was approved by Henri Verbeet. -- https://gitlab.winehq.org/wine/vkd3d/-/merge_requests/271
participants (4)
-
Alexandre Julliard (＠julliard) -
Giovanni Mascellani
-
Giovanni Mascellani (＠giomasce)
-
Henri Verbeet (＠hverbeet)