C++ Matrix
Testing C libraries can be a challenging and time-consuming task, especially when considering the various compiler versions, error behaviors, and platform dependencies. A library that works flawlessly with one compiler might encounter issues when compiled with a different version or on a different platform. This is where the C Library Test Matrix GitHub Action comes to the rescue.
The C Library Test Matrix GitHub Action is designed to automate the generation of a comprehensive test matrix for your C libraries given their requirements. It simplifies the process of testing your libraries across multiple compiler versions and platforms, reducing the manual effort and potential for human error.
With this action, you can define a set of compiler versions, target platforms, standard requirements, and factors that you want to test your C++ library against. The action then automatically generates a test matrix by combining the specified compilers and platforms into a fractional factorial design, enabling you to run tests in a systematic and efficient manner.
This action can ensure that your C++ library is thoroughly tested across a wide range of compiler versions and platforms. This approach significantly increases the chances of catching compatibility issues early on and helps in delivering a robust and reliable library that works across different environments.
-
Automated test matrix generation: Define a list of compiler versions and platforms, and let the action handle the creation of the test matrix.
-
Easy integration: Simply include the C++ Library Test Matrix GitHub Action in your workflow configuration to start generating the test matrix.
-
Customization options: Fine-tune the test matrix generation process by specifying additional compiler flags, build configurations, or test suites.
-
Comprehensive test coverage: Ensure your C++ library is thoroughly tested across multiple compiler versions and platforms to identify and address compatibility issues.
Get started with the C Library Test Matrix GitHub Action today and streamline your C library testing process. Say goodbye to manual matrix creation and embrace automated, systematic testing to deliver high-quality C++ libraries that work seamlessly across different compilers and platforms.
Usage
The action should be run as extra initial “setup” job in your workflow. The job will run the action and
output matrix, which is a JSON string containing the whole matrix. This matrix should be output of your
first setup job.
In your second build job, you can attribute the entire matrix to the strategy.matrix.include parameter
and create your workflow as usual with the parameters from the matrix:
jobs:
cpp-matrix:
runs-on: ubuntu-latest
name: Generate Test Matrix
outputs:
matrix: ${{ steps.cpp-matrix.outputs.matrix }}
steps:
- name: Generate Test Matrix
uses: alandefreitas/cpp-actions/cpp-matrix@v1.4.0
id: cpp-matrix
with:
standards: '>=11'
build:
needs: cpp-matrix
strategy:
fail-fast: false
matrix:
include: fromJSON(needs.cpp-matrix.outputs.matrix)
# use matrix entries
name: ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
container: ${{ matrix.container }}
steps:
- name: Clone library
uses: actions/checkout@v3
- name: Setup C++ Compiler
uses: alandefreitas/cpp-actions/setup-cpp@v1.4.0
id: setup-cpp
with:
compiler: ${{ matrix.compiler }}
version: ${{ matrix.version }}
- name: CMake Workflow
uses: ./cmake-workflow
with:
cc: ${{ steps.setup-cpp.outputs.cc || matrix.cc }}
cxx: ${{ steps.setup-cpp.outputs.cxx || matrix.cxx }}
cxxstd: ${{ matrix.cxxstd }}
cxxflags: ${{ matrix.cxxstd }}
# And you've safely tested your C++ library just like that...Example
steps:
- name: Generate Test Matrix
uses: alandefreitas/cpp-actions/cpp-matrix@v1.4.0
id: cpp-matrix
with:
standards: '>=11'
factors: 'gcc Asan Shared No-Deps
msvc Shared x86
clang Time-Trace
mingw Shared
'Input Parameters
Parameter |
Description |
Default |
|
A list of compilers to be tested. Each compiler can be complemented with its semver version requirements to be tested. When the compiler version requirements are provided, the action will break the requirements into subsets of major versions to be tested. When no version is provided, the '*' semver requirement is assumed. The action can identifies subsets of compiler versions for GCC, Clang, and MSVC. For any other compilers, the version requirements will passthrough to the output. . |
|
|
A semver range describing what C standards should be tested. The compiler ranges are adjusted to only include compilers that support any subrange of these requirements. These requirements can include C standards as 2 or 4 digits versions, such as 11, 2011, 98, or 1998. 2 digit versions are normalized into the 4 digits form so that 11 > 98 (2011 > 1998). . |
|
|
The maximum number of standards to be tested with each compiler. For instance, if 'max-standards' is 2 and the compiler supports '11,14,17,20,23' given the in the standard requirements, the standards 20,23 will be tested by this compiler. . |
|
|
The factors to be tested with the latest versions of each compiler. For each factor in this list, the entry with the latest version of a compiler will be duplicated with an entry that sets this factor to true. Other entries will also include this factor as false. The following factors are considered special: 'asan', 'ubsan', 'msan', 'tsan', and 'coverage'. When these factors are defined in an entry, its 'ccflags', 'cxxflags', and 'linkflags' value are also modified to include the suggested flags for factor. . |
|
|
The factors to be tested with other versions of each compiler. Each factor in this list will be injected into a version of the compiler that is not the latest version. An entry with the latest version of the compiler will be duplicated with this factor if there are no entries left to inject the factor. Other entries will also include this factor as false. . |
|
|
Generate summary with the complete matrix. |
|
|
Trace commands executed by the action. |
|
Outputs
Output |
Description |
|
The test matrix is an array of dictionaries, where each entry represents a combination of compiler version and factors to be tested. Each entry in the test matrix dictionary contains the following key-value pairs: - - - - - - - - - - `cc`: The usual name of the C compiler executable. If using the `setup-cpp` action, its output should be used instead. - `cxxstd`: A list of standards that should be tested with this compiler version. This option considers the `max-standards` latest standards supported by each compiler in its subrange of `standards`. - `b2-toolset`: The usual name of the toolset to be used in a b2 workflow. - `generator`: A CMake generator recommended to run the CMake workflow. - `build-type`: A build type recommended to test this entry. This is usually `Release`, unless some special factor that requires `Debug` is defined. - `ccflags`: The recommended C flags to be used by this entry. It reflects the values of special factors, such as sanitizers, coverage, and time-trace. - `cxxflags`: The recommended C flags to be used by this entry. It reflects the values of special factors, such as sanitizers, coverage, and time-trace. - - |