C++ Matrix

The C++ Matrix Action is designed to automate the generation of a comprehensive test matrix for your C++ libraries given their requirements. It simplifies the process of defining the contract of what platforms your library supports and testing your project across a proper selected subset of revelant compiler versions and platforms.

With this action, you can define a set of requirements to test your C++ library. It will then generate a test matrix by combining the specified requirements into a fractional factorial design to ensure all proper combinations are tested in a systematic manner. This approach significantly increases the chances of catching compatibility issues early on and helps in delivering a robust and reliable library to users.

Usage

The action should be run as extra initial “setup” job in your workflow. The job will run the action and output matrix, which is a JSON string containing the whole matrix. This matrix should be output of your first setup job.

In your second build job, you can attribute the entire matrix to the strategy.matrix.include parameter and create your workflow as usual with the parameters from the matrix:

jobs:
  cpp-matrix:
    runs-on: ubuntu-latest
    name: Generate Test Matrix
    outputs:
      matrix: ${{ steps.cpp-matrix.outputs.matrix }}
    steps:
      - name: Generate Test Matrix
        uses: alandefreitas/cpp-actions/cpp-matrix@v1.9
        id: cpp-matrix
        with:
          standards: '>=11'
  build:
    needs: cpp-matrix
    strategy:
      fail-fast: false
      matrix:
        include: fromJSON(needs.cpp-matrix.outputs.matrix)

    # use matrix entries
    name: ${{ matrix.name }}
    runs-on: ${{ matrix.runs-on }}
    container: ${{ matrix.container }}

    steps:
      - name: Clone library
        uses: actions/checkout@v3

      - name: Setup C&#43;&#43; Compiler
        uses: alandefreitas/cpp-actions/setup-cpp@v1.9
        id: setup-cpp
        with:
          compiler: ${{ matrix.compiler }}
          version: ${{ matrix.version }}

      - name: CMake Workflow
        uses: ./cmake-workflow
        with:
          cc: ${{ steps.setup-cpp.outputs.cc || matrix.cc }}
          cxx: ${{ steps.setup-cpp.outputs.cxx || matrix.cxx }}
          cxxstd: ${{ matrix.cxxstd }}
          cxxflags: ${{ matrix.cxxstd }}

      # And you've safely tested your C&#43;&#43; library just like that...

Factors

The input parameters allow factors to be defined for each compiler. These factors can be defined as latest factors (which are duplicated and applied to the latest version of each compiler) or as regular factors (which are injected into intermediary versions of each compiler).

See the Factors Section for a better understanding of the motivation for these factor types.

Matrix entries

The values in each matrix entry include a few categories of keys: the main keys, auxiliary keys, factor keys, and suggestions.

The main keys include information such as the compiler name and version. Auxiliary keys include information that is useful for filtering and sorting the matrix, such as information about the relative position of an entry in the matrix. Factor keys include information about the factors that might have been applied to the entry.

Suggestions include opinionated recommendations for other parameters of your workflow as a starting point for your test matrix. Some examples of keys including suggestions are runs-on, container, generator`, b2-toolset, build-type, ccflags, cxxflags, env, and install.

These suggestions often need to be customized and this could not be different, since there is a myriad of ways in which libraries should be tested. Common ways to customize these are:

Ignoring suggestions that are not applicable to your use case
Using the corresponding action input to customize the value of these fields (see action input parameters)
Use GitHub Actions Expressions to generate new values from existing values
Use a custom bash step to generate new values from existing values
Create a custom script to read the complete matrix and generate a new matrix with the desired values

For instance, if the recommendation of running a workflow in the container ubuntu:22.04 is not appropriate because you’re going to use a container that already has your dependencies, you could set the runs-on input parameter to:

- name: Generate Test Matrix
  # ...
  with:
    # ...
    containers: |
       gcc: my-ubuntu:22.04-container

This would replace the recommendation for ubuntu:22.04 with my-ubuntu:22.04-container for all entries with the compiler gcc.

In some cases, you could use a expression directly in the workflow to use a different value for the container:

# In your main workflow
containers: ${{ matrix.runs-on == 'ubuntu-22.04' && 'my-ubuntu:22.04-container' || matrix.container }}

This would replace the recommendation for ubuntu:22.04 with my-ubuntu:22.04-container for all entries.

If you’re familiar with GitHub Actions and bash, you can also use a custom bash step to generate new values from existing values or write a complete script to customize the matrix.

If none of these options is enough for the library requirements, the action also prints the complete matrix in YAML format, so it can be copy/pasted into the workflow as a starting point to be customized.

Extra values in entries

In many cases, we need to reuse GitHub expressions to generate new values based on existing values in the matrix. For instance, consider an entry in the matrix that has the x86 factor set to true.

Assume we are using b2 and should set address-model=32 for this entry. Otherwise, we should set address-model=64. With GitHub expressions, we can generate this value based on the x86 factor:

b2 address-model={{ matrix.x86 && '32' || '64' }}

If this pattern needs to be repeated in many entries, it might become cumbersome. In this case, the action allows the user to define extra values to be injected in each entry of the matrix. These values can be used to generate new values based on existing values in the matrix with handlebars. You could set the following extra values in each entry of the matrix:

extra-values: |
  address-model: \{{#if x86}}32\{{else}}64\{{/if}}

# ...
run: |
  b2 address-model={{ matrix.address-model }}

extra-values: |
  address-model-arg: address-model=\{{#if x86}}32\{{else}}64\{{/if}}

# ...
run: |
  b2 {{ matrix.address-model-arg }}

The syntax for each entry in the extra-values input is a key-value pair, where the key is the name of the value to be injected in the matrix and the value is a handlebars expression that generates the value based on other values in the matrix. The handlebars expression can access previous values in the matrix, including other extra values.

Besides referencing other values in the matrix, the handlebars expressions can also use the built-in helpers provided by Handlebars and the predefined helpers defined by the action:

String Helpers for Extra Values

Helper

Description

lowercase

Converts the value to lowercase

uppercase

Converts the value to uppercase

contains

Checks if the value contains a substring (case-sensitive)

startsWith / starts-with

Checks if the value starts with a substring

endsWith / ends-with

Checks if the value ends with a substring

substr

Extracts a substring: substr str start end

replace

Replaces all occurrences: replace str search replacement

replaceFirst

Replaces first occurrence only

indexOf

Returns index of substring, or -1 if not found

lastIndexOf

Returns last index of substring, or -1 if not found

split

Splits string into array: split str delimiter

trim

Removes leading/trailing whitespace

trimLeft

Removes leading whitespace

trimRight

Removes trailing whitespace

capitalize

Capitalizes first character

titlecase

Capitalizes first letter of each word

camelcase

Converts to camelCase

pascalcase

Converts to PascalCase

snakecase

Converts to snake_case

kebabcase

Converts to kebab-case

reverse

Reverses a string (or array)

Case-Insensitive String Helpers

These helpers match GitHub Actions expression behavior (case-insensitive):

Helper

Description

icontains

Case-insensitive substring check

istartsWith / istarts-with

Case-insensitive prefix check

iendsWith / iends-with

Case-insensitive suffix check

Logical Helpers

Helper

Description

and

Logical AND (supports multiple arguments)

or

Logical OR (supports multiple arguments)

not

Logical NOT

select

Ternary operator: select condition trueValue falseValue

Comparison Helpers

Helper

Description

eq

Strict equality (===)

ieq

Case-insensitive equality

ne

Inequality (!==)

ine

Case-insensitive inequality

lt

Less than (<)

le

Less than or equal (<=)

gt

Greater than (>)

ge

Greater than or equal (>=)

Math Helpers

Helper

Description

add

Addition: add a b

sub

Subtraction: sub a b

mul

Multiplication: mul a b

div

Division: div a b

mod

Modulo: mod a b

abs

Absolute value

floor

Round down

ceil

Round up

round

Round to nearest integer

min

Minimum of values: min a b c …

max

Maximum of values: max a b c …

pow

Power: pow base exponent

Array Helpers

Helper

Description

join

Joins array elements: join array delimiter

first

Returns first element

last

Returns last element

nth

Returns nth element (0-indexed): nth array index

length

Returns length of array or string

slice

Extracts portion: slice array start end

sort

Sorts array alphabetically

includes

Checks if array contains value

Type Checking Helpers

Helper

Description

isString

Returns true if value is a string

isNumber

Returns true if value is a number

isArray

Returns true if value is an array

isEmpty

Returns true if value is empty string, null, undefined, or empty array

Conversion Helpers

Helper

Description

toNumber

Converts string to number

toJSON

Converts value to JSON string (GitHub Actions compatible)

fromJSON

Parses JSON string to value (GitHub Actions compatible)

Utility Helpers

Helper

Description

default

Returns default if value is falsy: default value fallback

coalesce

Returns first non-null/undefined value: coalesce a b c

format

Template substitution: format "Hello {0}" name (GitHub Actions compatible)

Ordering entries

Entries are ordered in the matrix according to the following criteria:

1) Latest versions of each compiler 2) Compilers with single versions 3) Oldest versions of each compiler 4) Entries with factors 5) Intermediary versions of each compiler

The input parameter generate-summary can be used to generate a summary of the matrix with the entries ordered according to these criteria.

Semver ranges

The requirement parameters support semver ranges, including range disjunctions. For instance, consider the default compiler range clang >=3.8, which might test your library with too many versions of clang. You could remove intermediary versions of clang with the range clang >=3.8 <5 || >=10, which ensures old versions and recent versions are covered, while eliminating intermediary versions.

Semver ranges can also help when a bug is found in a specific compiler version. Consider you’re testing gcc >=4.8 and someone has reported a bug in GCC 8.1. Now you want to keep track of GCC 8.1 specifically. The semver range gcc >=4.8 <8 || 8.1 || >=9 would ensure version GCC 8.1 specifically, and not any other version is the range >=8.0.0 <9.0.0 is tested.

The action still accepts open ranges to test the latest versions of a compiler. However, additional actions might be needed to test the latest compilers, which may not be implemented in this version of the project. Therefore, users are still recommended to avoid open ranges or to update the action’s version to support the latest compiler versions.

Dynamic matrices

In some cases, it might be useful to test different matrices based on the conditions of the workflow. These conditions might be the event_type and the types of files changed in the commit triggering the workflow. The workflows might vary between disabling tests for changes that don’t affect these certain files, running a reduced subset of tests for less important changes, or enabling extra tests, such as documentation tests in case only these files have been changed. This might be useful in terms of performance, costs, and safety.

While this is hard to achieve with hard-coded matrices, this action makes variable matrices very easy to achieve. Simply define a previous step to determine the matrix parameters and use it to generate the matrix. For instance, the compilers parameters could be replaced from gcc >=4.8 to gcc >=4.8 <6.0 ${{ github.event_name == 'push' && '|| >=6.0 <10' }} || >=11 to only test intermediary GCC versions >=6.0 <10 when in push events.

A more powerful strategy to save resources is analyzing files changed in the workflow and considering these files to determine what the matrix should be according to project conventions.

Patching Node on old containers

If you need to run tests on old compilers, the matrix entries might include old containers. GitHub Actions stopped supporting many old containers since it moved to Node 20.

A workaround is to set the volumes key of the container object so that the user can provide its own Node installation.

This action will provide these suggestions in the container key of the matrix entries. However, the user should still include a step in the workflow to install the required Node version in the /__e/node20 directory.

Next steps

After creating the test matrix, the next step in your workflow should usually be Setup C++.

Example

steps:
- name: Generate Test Matrix
  uses: alandefreitas/cpp-actions/cpp-matrix@v1.9
  id: cpp-matrix
  with:
    standards: '>=14'
    compilers: 'gcc >=4.8 <6 || >=9

      clang >=3.8 <6 || >11

      msvc >=14

      apple-clang *

      mingw *

      clang-cl *

      '
    subrange-policy: 'msvc: one-per-minor

      '
    latest-factors: 'gcc Coverage TSan UBSan Fetch-Content

      clang Fetch-Content

      '
    factors: 'gcc ASan Shared No-Deps

      msvc Shared x86

      clang Time-Trace ASan+UBSan

      mingw Shared

      '
    force-factors: 'mingw: No-Deps

      '
    extra-values: 'vcpkg-packages: \{{#if (not fetch-content)}}fmt\{{/if\}}

      address-model: \{{#if (eq arch ''x86'')}}32\{{else}}64\{{/if\}}

      '
    github-token: ${{ secrets.GITHUB_TOKEN }}

Input Parameters

Parameter

Description

Default

trace-commands

Trace commands executed by the workflow.

false

compilers

A list of compilers to be tested. Each compiler can be complemented with its semver version requirements to be tested.

When the compiler version requirements are provided, the action will break the requirements into subsets of major versions to be tested. When no version is provided, the '*' semver requirement is assumed. The action can identifies subsets of compiler versions for GCC, Clang, and MSVC. For any other compilers, the version requirements will passthrough to the output.

gcc >=4.8

clang >=3.8

msvc >=14

apple-clang *

mingw *

clang-cl *

subrange-policy

The policy to be used to break the compiler version requirements into sub-ranges of versions.

For instance, if the compiler requirements are gcc >=4.8, the action will typically generate entries that satisfy gcc >=4.8 <5, gcc >=5 <6, gcc >=6 <7, and so on, because the default policy is one-per-major.

The policy can be one-per-major or one-per-minor.

This input can be a single value for all compilers or a multi-line list of compiler-specific policies.

Another policy is to break into major versions when the range contains multiple major versions and into minor versions when the range contains multiple minor versions. The name of this policy is one-per-major-minor.

: one-per-major

standards

A semver range describing what C standards should be tested. For instance, `>=11` indicates that the library should be tested with C11 and later standards.

These requirements can include C standards as 2 or 4 digits versions, such as 11, 2011, 98, or 1998. 2 digit versions are normalized into the 4 digits form so that 11 > 98 (2011 > 1998).

The action will generate entries for each compiler version that satisfies the requirements. The compiler ranges defined in `compilers` are adjusted to only include compilers that support any subrange of these requirements. Compilers that don't support any of the standards in the range will be excluded from the matrix.

Each entry in the matrix will include the `cxxstd` key with a list of standards to be tested with that compiler version. This list will include the `max-standards` latest standards supported by the compiler specified in that entry. For instance, if `max-standards` is `3` and the compiler supports '11,14,17,20,23' given the the `standards` requirement `>=11`, the `cxxstd` field of the entry will include the standards `20,23` will be tested by this compiler. This allows the matrix to be more focused on the latest standards supported by each compiler (the ones that are more likely to be contain compatibility issues) while still testing all standards required by the library in the matrix.

It's very common for compilers to not fully comply with the standards they claim to support, even for the old standards. The criteria used by this action for determining if a compiler supports a standard is based on the whether the compiler claims to support the standard by providing a corresponding `-std=cXX` flag to enable the standard. This criteria is easy to follow, minimizes surprises, covers the most common bugs, and ensures users the library is compliant with all standard flags supported by the compilers.

>=11

max-standards

The maximum number of standards to be tested with each compiler.

For instance, if 'max-standards' is 2 and the compiler supports '11,14,17,20,23' given the in the standard requirements, the standards 20,23 will be tested by this compiler.

2

latest-factors

The factors to be tested with the latest versions of each compiler. For each factor in this list, the entry with the latest version of a compiler will be duplicated with an entry that sets this factor to true.

Other entries will also include this factor as false.

The following factors are considered special: 'asan', 'ubsan', 'msan', 'tsan', and 'coverage'. When these factors are defined in an entry, its 'ccflags', 'cxxflags', and 'linkflags' value are also modified to include the suggested flags for factor.

gcc Coverage TSan UBSan

factors

The factors to be tested with other versions of each compiler. Each factor in this list will be injected into a version of the compiler that is not the latest version. An entry with the latest version of the compiler will be duplicated with this factor if there are no entries left to inject the factor.

Other entries will also include this factor as false.

gcc Asan Shared

msvc Shared x86

clang Time-Trace

mingw Shared

combinatorial-factors

The factors to be tested with all combinations of other factors. When combinatorial factors are defined, for each entry in the matrix, a new entry will be created with the factors in this list set to true.

For instance, if the library can be built both in "Standalone" mode and with dependencies, the factor 'Standalone' can be added to this list to duplicate all entries. Each copy would include a "Standalone" factor set to true or false.

Typically, it is advisable to steer clear of combinatorial factors to prevent a combinatorial explosion. It’s usually better to only test the combinations that are expected to be used in practice and include an extra steps in the workflow to test any combinatorial factors.

For instance, if the library can be built both in "Standalone" mode and with dependencies, its workflow can simply include an extra step to also test the library in "Standalone" mode and keep the step to test the library with dependencies. This is usually safer and cheaper than duplicating the entire matrix to test all combinations of these factors and also allows steps to be skipped when the library is not expected to be built in a given mode. For instance, testing a library on Standalone mode might not be necessary when the library is being tested with intermediary compilers.

force-factors

A multi-line list of factor flags to be injected with each range of compiler version even if the entry doesn’t have the usual requirements to have that factor.

Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <build-type>

For instance, gcc >=13 <14: Asan indicates that the flag asan will be included with any version of gcc in that range, even if the entry doesn’t have the usual requirements to have the Asan factor.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the build type to all versions of the compiler.

When the build type is unspecified, the action will infer the build type from the compiler name and its version.

extra-values

A multi-line list of key-value pairs to be injected in each entry of the matrix.

Each line has the format:

<key>: <value>

For instance, hash-key: value includes the hash-key with the value value in each entry of the matrix. This means it can later be accessed more easily via the matrix output of the action.

The values also support Handlebars expressions, which can be used to generate values based on other values in the entry. For instance,

hash-key: {{ compiler }}-{{ version }}

would generate a hash-key with the value gcc-11.1 for an entry with the compiler gcc and version 11.1. Previous extra values in the entry can also be accessed via the matrix output of the action. For instance:

cache-key: cache-{{ hash-key }}.

runs-on

A multi-line list of github runner images to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <github-runner-image>

For instance, gcc >=13.1: ubuntu-latest indicates that the runner image ubuntu-latest should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the runner image to all versions of the compiler.

When the runner image is specified, a container is only be suggested for the entries if the container option for that compiler version is also specified.

When the runner image is unspecified, the action will infer the runner image and potentially a container from the compiler name and its version.

containers

A multi-line list of docker containers to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <docker-container>

For instance, gcc >=13.1: ubuntu:22.04 indicates that the docker container ubuntu:22.04 should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the container to all versions of the compiler.

When the container is specified for that compiler version and the runs-on option is not, an ubuntu image is suggested for the entry to run the container.

When the container is unspecified, the action can still infer a container for the compiler version according to the rules defined in the use-containers option.

generators

A multi-line list of cmake generators to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <cmake-generator>

For instance, gcc >=13.1: Ninja indicates that the cmake generator Ninja should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the generator to all versions of the compiler.

When the generator is unspecified, the action will infer the generator from the compiler name and its version.

generator-toolsets

A multi-line list of cmake generator toolsets to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <cmake-generator-toolset>

For instance, clang-cl *: ClangCL indicates that the cmake generator toolset ClangCL should be used to test clang-cl with any version.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the generator to all versions of the compiler.

When the generator toolset is unspecified, the action will infer the generator toolset from the compiler name and its version.

b2-toolsets

A multi-line list of b2 toolsets to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <b2-toolset>

For instance, gcc >=13.1: gcc indicates that the b2 toolset gcc-13 should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the toolset to all versions of the compiler.

When the toolset is unspecified, the action will infer the toolset from the compiler name and its version.

ccflags

A multi-line list of C compiler flags to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <ccflags>

For instance, gcc >=13.1: -O3 indicates that the C compiler flag -O3 should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the flag to all versions of the compiler.

When the flag is unspecified, the action will infer the flag from the compiler name and its version.

cxxflags

A multi-line list of C compiler flags to be used with each range of compiler version. Each line has the format:

`<compiler-name>[ <compiler-range|compiler-factor>]: <cxxflags>`

For instance, `gcc >=13.1: -O3` indicates that the C compiler flag -O3 should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the flag to all versions of the compiler.

When the flag is unspecified, the action will infer the flag from the compiler name and its version.

install

A multi-line list of packages to be installed with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <packages>

For instance, gcc >=13.1: build-essential indicates that the package build-essential should be installed to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the package to all versions of the compiler.

When the package is unspecified, the action will infer the package from the compiler name and its version.

triplets

A multi-line list of triplets to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <triplet>

For instance, gcc >=13.1: x86_64-linux-gnu indicates that the triplet x86_64-linux-gnu should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the triplet to all versions of the compiler.

When the triplet is unspecified, the action will infer the triplet from the compiler name and its version.

build-types

A multi-line list of build types to be used with each range of compiler version. Each line has the format:

<compiler-name>[ <compiler-range|compiler-factor>]: <build-type>

For instance, gcc >=13.1: Release indicates that the build type Release should be used to test gcc with any version in the semver range >=13.1.

Omitting <compiler-range|compiler-factor> is equivalent to it being set to * and will apply the build type to all versions of the compiler.

When the build type is unspecified, the action will infer the build type from the compiler name and its version.

default-build-type

The default build type to suggest for entries without a specific build type.

Release

sanitizer-build-type

Determine the default build type to suggest when testing with sanitizers.

RelWithDebInfo

x86-build-type

Determine the default build type to suggest when testing with x86.

Release

use-containers

Determine whether to use containers whenever possible to run the tests.

By using containers for all jobs, the workflow can be more stable and reproducible. For instance, without containers an existing workflow cannot start to fail because of a change in the GitHub runner environments.

However, this comes at a cost of initial setup time. Some existing workflows can also break when moving to containers because existing assumptions about tools available in the runner environment are no longer valid.

When the value is false, the action will still use containers when needed. This may happen because the compiler is not available in the runner image or when there’s a reported conflict between compilers in the runner image.

true

output-file

The file to output the matrix as a JSON string.

This is useful when the matrix is too large to be printed in the logs or when the matrix needs to be saved for later use.

The file will be saved in the current working directory of the action.

matrix.json

log-matrix

Log the generated matrix as a JSON string.

The is useful for debugging purposes and when transitioning to a workflow that uses a hard-coded matrix.

true

generate-summary

Generate summary with the complete matrix.

true

warn-no-matches

Emit a GitHub warning whenever a compiler configuration results in zero matrix entries because no known versions can satisfy the requested version range and C++ standard requirements simultaneously. Leave this enabled to catch unintended gaps; set it to false if you deliberately provide unsatisfiable requirements and do not want the additional annotation.

true

sort-by-failure-rate

Sort matrix entries by historical failure rate.

The action fetches recent workflow run history and calculates the failure rate for each matrix entry based on job name matching. Entries with higher failure rates are sorted first (stable sort preserving existing order for equal rates).

This is useful when combined with max-parallel or limited runners, as it ensures jobs most likely to fail run first, providing faster feedback.

When enabled, the summary table includes a "Failure Rate" column.

Requires github-token to be set for API access. If failure rates cannot be calculated (no token, no history, API errors), the action continues gracefully without error.

true

failure-rate-runs

Number of recent workflow runs to analyze when calculating failure rates.

Jobs for each run are fetched in parallel, so increasing this value has minimal impact on execution time.

Only used when sort-by-failure-rate is enabled.

100

github-token

GitHub token for API access when fetching workflow run history for failure rate calculation.

Required when sort-by-failure-rate is enabled. Pass secrets.GITHUB_TOKEN or github.token. Alternatively, set the GITHUB_TOKEN environment variable in your workflow step.

Outputs

Output

Description

matrix

The test matrix is an array of dictionaries, where each entry represents a combination of compiler version and factors to be tested.

Each entry in the test matrix dictionary contains key-value pairs in the following categories:

Basic fields:

- name: A suggested name for the job testing this entry

- compiler: Specifies the name of the compiler to be used for the test configuration. This can be used as input to the setup-cpp action.

- version: Specifies the version requirements of the compiler to be used for the test configuration. This can be used as input to the setup-cpp action.

- cxxstd: A list of standards that should be tested with this compiler version. This option considers the max-standards latest standards supported by each compiler in its subrange of standards.

- latest-cxxstd: The last standard listed in cxxstd as a convenience variable

Auxiliary:

- major, minor, patch: Specifies the version components of the compiler whenever the whole range includes a single major, minor, or patch.

- has_major, has_minor, has_patch: Determines if the corresponding version component of the compiler is available representing all versions in the range.

- is-latest: Specifies whether the entry version requirement is the latest version among the test configurations.

- is-main: Specifies whether the entry version requirement is the latest version among the test configurations without any factors applied.

- is-earliest: Specifies whether the entry version requirement is the earliest version among the test configurations.

- is-intermediary: Specifies whether the entry version requirement is neither the earliest nor the latest version among the test configurations.

- has-factors: Specifies whether the entry has any factors applied.

- is-no-factor-intermediary: Specifies whether the entry is an intermediary version without any factors applied.

- is-container: Specifies whether the entry has a container suggested.

Factors:

- <factors>…: Provides additional factors or attributes associated with the test configuration as defined by the factors inputs. These usually include variant build configurations spread among the entries, such as asan, coverage, and shared libraries. For instance, if the Asan factor is applied to an entry, the entry will define the asan key with the value true and all other entries will define the asan key with the value false.

Suggestions:

- runs-on: A suggested github runner image name for the job testing this entry

- container: A suggested docker container for the job testing this entry

- cxx: The usual name of the C++ compiler executable. If using the setup-cpp action, its output should be used instead.

- cc: The usual name of the C compiler executable. If using the setup-cpp action, its output should be used instead.

- b2-toolset: The usual name of the toolset to be used in a b2 workflow.

- generator: A CMake generator recommended to run the CMake workflow.

- build-type: A build type recommended to test this entry. This is usually Release, unless some special factor that requires Debug is defined.

- ccflags: The recommended C flags to be used by this entry. It reflects the values of special factors, such as sanitizers, coverage, and time-trace.

- cxxflags: The recommended C++ flags to be used by this entry. It reflects the values of special factors, such as sanitizers, coverage, and time-trace.

- env: A dictionary of environment variables to be set for this entry.

- install: The recommended packages to be installed before running the workflow. This includes packages such as build-essential for ubuntu containers and lcov for coverage entries.