cmake-re supports the --distributed --build command line flags, scaling CMake builds to thousands of Jobs on RBE build clusters like the EngFlow platform.

This supports fully distributed:

• Compiling
• Archiving
• Linking

Remote Execution API (RE-API)

The RE API (Remote Execution API) is an open and popular remote execution protocol to enable build and test actions to be executed on remote servers instead of local machines, making builds faster and more scalable.

Born out of Google's Blaze build system, it is an open gRPC protocol and it is now getting wider adoption outside of Blaze/Bazel. cmake-re implements it for CMake together with our own fork of reclient to remote invocations of compiler, archiver, linker, tests and custom commands.

Usage

Authenticate with an mTLS Certificate

• Head in your browser to https://<cluster-address>/gettingstarted
• Download the mTLS certicates by clicking the Button:

This will download a file named engflow-mTLS.zip containing an:

• engflow.crt
• engflow.key (This is the private key, never share)

Setup Certificates for cmake-re

# Setup RBE Cluster for L2 caching and remote execution
export RBE_service=<cluster-address:port>
export RBE_tls_client_auth_key=/path/to/engflow.key
export RBE_tls_client_auth_cert=/path/to/engflow.crt

Build --distributed your CMake project

In order to run a distributed build, you will need to declare the environment in which it runs. This is given in the form of a CMake RE Environment Description.

Mainly the .pkr.js file aside the CMAKE_TOOLCHAIN_FILE declares which container image to use when running the build remotely.

Note on environments

A CMake RE Environment Description, essentially is:

• • A CMAKE_TOOLCHAIN_FILE, e.g. environment/linux.cmake
• • An accompanying .pkr.js and Dockerfile, e.g. environments/linux.pkr.js/, environments/linux.pkr.js/linux.Dockerfile

You can use an existing default environment or specify your own.

The best experience currently is with --host --distributed builds. This requires you to have an --host build environment matching remote execution. You can do so by starting the build from within the same container as the one configured in the .pkr.js file.

# Disable L1 caching operations
export TIPI_CACHE_CONSUME_ONLY=ON
export TIPI_CACHE_FORCE_ENABLE=OFF

# configure
cmake-re --host -S . -B ./build  -GNinja -DCMAKE_TOOLCHAIN_FILE=environments/linux.cmake
# build
cmake-re --host --distributed --build ./build -j1000

RBE FAQ

What if I require different local and remote environments?

CMake RE makes it particularly hard and will warn you about it when it detects any mismatches.

🧪 Advanced debugging and power-users: it's possible to override the mapping between the local environment cmake-re uses and the one use for remote build execution. This can be done by setting the RBE_platform environment variable:

# official linux environment for cmake-re v0.0.80
export RBE_platform=container-image=docker://tipibuild/tipi-ubuntu@sha256:5206328aa68f666b572c4e6ce1bf1b33731a01f36c3a1a4b9a003108f9370a42

✈️ Flight mode - How to continue working with a --distributed --build without an internet connection?

If one started working on a build tree with the --distributed mode but happens to have lost the internet connection (e.g. Working from a plane) one can disable the use of remote resources temporarily with:

export RBE_remote_disabled="true"

💡 Don't forget to ajdust the number of jobs -j used for the build, to avoid overloading your local machine, when building with RBE_remote_disabled.