[PATCH 0/2] MR4846: gitlab: Fix and optimize cache configuration.

[Rémi Bernon (＠rbernon)]

This partially reverts commit 67f2da2a8d1f219f528bf820b3d6002c5f34d5e4 which
broke ccache cache as the job configurations were overriding the default
cache configuration from .wine-build instead of extending it.

It then also updates the ccache cache config to add a key and try to avoid cache
invalidation between jobs with different compilers.

[PATCH 1/2] gitlab: Use a common configure cache key prefix.

[Rémi Bernon]

From: Rémi Bernon

Partially reverts commit 67f2da2a8d1f219f528bf820b3d6002c5f34d5e4 which
broke ccache cache as the job configurations were overriding the default
cache configuration from .wine-build instead of extending it.
---
 tools/gitlab/build-clang |  2 +-
 tools/gitlab/build.yml   | 30 +++++++++---------------------
 2 files changed, 10 insertions(+), 22 deletions(-)

diff --git a/tools/gitlab/build-clang b/tools/gitlab/build-clang
index b93bdf146f3..3ed87b3b173 100755
--- a/tools/gitlab/build-clang
+++ b/tools/gitlab/build-clang
@@ -9,7 +9,7 @@ set -Eeuxo pipefail
 ./tools/make_makefiles
 autoreconf -f

-cd build
+cd build64
 ../configure -q -C --enable-archs=i386,x86_64,aarch64 --with-mingw=clang
 make -s -j$(nproc)
 cd ..
diff --git a/tools/gitlab/build.yml b/tools/gitlab/build.yml
index 9337d6c3b43..9e5ce467e96 100644
--- a/tools/gitlab/build.yml
+++ b/tools/gitlab/build.yml
@@ -9,6 +9,13 @@
   cache:
     - paths:
         - ccache/
+    - key:
+        files:
+          - configure.ac
+        prefix: $CI_JOB_NAME-config
+      paths:
+        - build32/config.cache
+        - build64/config.cache
   before_script:
     - export BASEDIR="$PWD"
     - export CCACHE_BASEDIR="$BASEDIR"
@@ -28,13 +35,6 @@ build-linux:
     expire_in: 1 day
     paths:
       - usr/local/
-  cache:
-    - key:
-        files:
-          - configure.ac
-      paths:
-        - build32/config.cache
-        - build64/config.cache
   script:
     - mkdir -p build32 build64
     - git rebase $CI_MERGE_REQUEST_DIFF_BASE_SHA --exec ./tools/gitlab/build-linux
@@ -45,14 +45,8 @@ build-clang:
   extends: .wine-build
   rules:
     - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
-  cache:
-    - key:
-        files:
-          - configure.ac
-      paths:
-        - build/config.cache
   script:
-    - mkdir -p build
+    - mkdir -p build32 build64
     - ./tools/gitlab/build-clang

build-mac:
@@ -61,13 +55,6 @@ build-mac:
     - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
   tags:
     - mac
-  cache:
-    - key:
-        files:
-          - configure.ac
-      paths:
-        - build32/config.cache
-        - build64/config.cache
   artifacts:
     when: on_failure
     paths:
@@ -107,6 +94,7 @@ build-daily-linux:
     paths:
       - usr/local/
   script:
+    - mkdir -p build32 build64
     - ./tools/gitlab/build-linux

build-daily-winetest:

[PATCH 2/2] gitlab: Add key to the jobs ccache cache config.

[Rémi Bernon]

From: Rémi Bernon

To avoid jobs with different compilers overwriting each other.

As this will cause a different key to be used between build-daily-linux
and the MR jobs, also add $CI_MERGE_REQUEST_DIFF_BASE_SHA to the key to
try sharing caches across MR with the same base commit.
---
 tools/gitlab/build.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/gitlab/build.yml b/tools/gitlab/build.yml
index 9e5ce467e96..f5c0e39ccbe 100644
--- a/tools/gitlab/build.yml
+++ b/tools/gitlab/build.yml
@@ -7,7 +7,8 @@
   variables:
     GIT_DEPTH: 0
   cache:
-    - paths:
+    - key: $CI_JOB_NAME-ccache-$CI_MERGE_REQUEST_DIFF_BASE_SHA
+      paths:
         - ccache/
     - key:
         files:

[Rémi Bernon (＠rbernon)]

Fwiw I tried using fallback cache keys to share ccache between the daily build and the MR builds, but this doesn't seem to work well, especially with the macOS runner which apparently is too old and doesn't support `cache:fallback_keys` config.

[Rémi Bernon (＠rbernon)]

Arguably this could avoid the `$CI_MERGE_REQUEST_DIFF_BASE_SHA` suffix for the cache key, which would make the cache persist across different rebases and MRs.

With the suffix we can be sure that when the Gitlab cache is hit, the ccache is going to be quite close, and rebuild will be quick. However it will hit less often and only if an MR has previously been built against the same base commit.

Without the suffix the Gitlab cache will hit more often, but possibly with completely unrelated objects in the ccache, and the last MR being built will update the ccache with its objects, regardless of whether they are based on recent code or not.

Depending on the ccache size, it may be worth not using it, idk.