[PATCH 0/2] MR4846: gitlab: Fix and optimize cache configuration.
This partially reverts commit 67f2da2a8d1f219f528bf820b3d6002c5f34d5e4 which broke ccache cache as the job configurations were overriding the default cache configuration from .wine-build instead of extending it. It then also updates the ccache cache config to add a key and try to avoid cache invalidation between jobs with different compilers. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/4846
From: Rémi Bernon <rbernon(a)codeweavers.com> Partially reverts commit 67f2da2a8d1f219f528bf820b3d6002c5f34d5e4 which broke ccache cache as the job configurations were overriding the default cache configuration from .wine-build instead of extending it. --- tools/gitlab/build-clang | 2 +- tools/gitlab/build.yml | 30 +++++++++--------------------- 2 files changed, 10 insertions(+), 22 deletions(-) diff --git a/tools/gitlab/build-clang b/tools/gitlab/build-clang index b93bdf146f3..3ed87b3b173 100755 --- a/tools/gitlab/build-clang +++ b/tools/gitlab/build-clang @@ -9,7 +9,7 @@ set -Eeuxo pipefail ./tools/make_makefiles autoreconf -f -cd build +cd build64 ../configure -q -C --enable-archs=i386,x86_64,aarch64 --with-mingw=clang make -s -j$(nproc) cd .. diff --git a/tools/gitlab/build.yml b/tools/gitlab/build.yml index 9337d6c3b43..9e5ce467e96 100644 --- a/tools/gitlab/build.yml +++ b/tools/gitlab/build.yml @@ -9,6 +9,13 @@ cache: - paths: - ccache/ + - key: + files: + - configure.ac + prefix: $CI_JOB_NAME-config + paths: + - build32/config.cache + - build64/config.cache before_script: - export BASEDIR="$PWD" - export CCACHE_BASEDIR="$BASEDIR" @@ -28,13 +35,6 @@ build-linux: expire_in: 1 day paths: - usr/local/ - cache: - - key: - files: - - configure.ac - paths: - - build32/config.cache - - build64/config.cache script: - mkdir -p build32 build64 - git rebase $CI_MERGE_REQUEST_DIFF_BASE_SHA --exec ./tools/gitlab/build-linux @@ -45,14 +45,8 @@ build-clang: extends: .wine-build rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' - cache: - - key: - files: - - configure.ac - paths: - - build/config.cache script: - - mkdir -p build + - mkdir -p build32 build64 - ./tools/gitlab/build-clang build-mac: @@ -61,13 +55,6 @@ build-mac: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' tags: - mac - cache: - - key: - files: - - configure.ac - paths: - - build32/config.cache - - build64/config.cache artifacts: when: on_failure paths: @@ -107,6 +94,7 @@ build-daily-linux: paths: - usr/local/ script: + - mkdir -p build32 build64 - ./tools/gitlab/build-linux build-daily-winetest: -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/4846
From: Rémi Bernon <rbernon(a)codeweavers.com> To avoid jobs with different compilers overwriting each other. As this will cause a different key to be used between build-daily-linux and the MR jobs, also add $CI_MERGE_REQUEST_DIFF_BASE_SHA to the key to try sharing caches across MR with the same base commit. --- tools/gitlab/build.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/gitlab/build.yml b/tools/gitlab/build.yml index 9e5ce467e96..f5c0e39ccbe 100644 --- a/tools/gitlab/build.yml +++ b/tools/gitlab/build.yml @@ -7,7 +7,8 @@ variables: GIT_DEPTH: 0 cache: - - paths: + - key: $CI_JOB_NAME-ccache-$CI_MERGE_REQUEST_DIFF_BASE_SHA + paths: - ccache/ - key: files: -- GitLab https://gitlab.winehq.org/wine/wine/-/merge_requests/4846
Fwiw I tried using fallback cache keys to share ccache between the daily build and the MR builds, but this doesn't seem to work well, especially with the macOS runner which apparently is too old and doesn't support `cache:fallback_keys` config. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/4846#note_57433
Arguably this could avoid the `$CI_MERGE_REQUEST_DIFF_BASE_SHA` suffix for the cache key, which would make the cache persist across different rebases and MRs. With the suffix we can be sure that when the Gitlab cache is hit, the ccache is going to be quite close, and rebuild will be quick. However it will hit less often and only if an MR has previously been built against the same base commit. Without the suffix the Gitlab cache will hit more often, but possibly with completely unrelated objects in the ccache, and the last MR being built will update the ccache with its objects, regardless of whether they are based on recent code or not. Depending on the ccache size, it may be worth not using it, idk. -- https://gitlab.winehq.org/wine/wine/-/merge_requests/4846#note_57434
participants (1)
-
Rémi Bernon