Tipi can detect and run tests and examples as specified in tipi build conventions.

Executing all tests

To have tipi run all tests after a successful build use the --test all command line parameter:

tipi . -t linux-cxx17 --test all

Executing specific tests

Example 1: Run only tests defined in the executable mytest

tipi . --test mytest

The parameter can take a regex pattern to fit more complex cases:

Example 2: Run tests starting with system and ending in smoke

tipi . --test "^system.+smoke$"

Note: You may use the <project-root>/.tipi.force-tests file to override --test - see §Test control files

Excluding specific tests

If you need to exclude a number of test executables matched by --test <all|pattern> you can use the --exclude-test parameters. This parameter can take a regex pattern to fit more complex cases:

Example: run all tests except those matching mytest1|mytest2

tipi . --test all --exclude-test "mytest1|mytest2"

Passing parameters to the test executable

Use the --test-args <value> command line parameter of tipi if you need to pass command line arguments to the test executable:

Example: disable the test "intro" output and use minimal logging in a doc-test based test:

tipi . -t linux-cxx17 --test test/mytest --test-args "--no-intro=true --minimal" 

Note: You may use the <project-root>/.tipi.test-args file to override --test-args - see §Test control files

Test control files

In order to provide additional flexibility for live build and continuous integration setups, tipi provides file-based facilities to control the values of --test and --test-args:

• <project-root>/.tipi.force-tests overrides the value of --test
  • enable tests even if no --test parameter was passed while staying in monitor mode
  • change the selected test executables in monitor mode
• <project-root>/.tipi.test-args overrides the value of --test-args
  • change the test executable parameters while staying in monitor mode
  • set default values / complex arguments used in testing

Example: Filter for test executables matching $system- (e.g. starting with "system-") and set the test arguments to the same values as in the example above.

In <project-root>/.tipi.force-tests :

^system-

In <project-root>/.tipi.test-args :

--no-intro=true
--minimal

Then the test:

tipi .

Note: any newline \n or CR+LF characters in the file content will be replaced by a single white space from the file content, so you may specify the passed parameters on distinct lines.

Test execution concurrency

By default tipi will execute the test executables using the same concurrency level as set by the -j / compile jobs parameter (which defaults to the number of hardware threads of the system CPU).

If you want to change this setting independently you can specify --test-jobs <N> with <N> the number of concurrently executed binaries.

Example 1: linear (one at a time) test execution (while compiling with default concurrency):

tipi . --test all --test-jobs 1

Example 2: linear (one at a time) test execution and compiling at -j4:

tipi . --test all --test-jobs 1 -j4

Note: in a remote build setting --test-jobs does not change the machine size used to execute the tests.