Have you ever sat in a restaurant and wondered: Why the heck do they already have their Cock Au Vin on the next table and my simple pasta is taking forever? If you're anything like us, you immediately apply for funding to set up a large scale experiment measuring preparation times for all combinations of recipes. It's, of course, the only reasonable thing to do.

Well, we can't help you with the funding. What we can offer you is a comfy kitchen bench to take a seat, observe and come up with a unified theory of bolognesio synthesis revolutionising the field. So, compose your recipe, lean back and observe...

Wait, did we say "kitchen bench"? We meant...

KitGenBench

KitGenBench is a Kit for Generating (Micro-)Benchmarks based on alpaka. As such, it is suitable to generate benchmarks on a wide variety of hardware architectures including all major CPU and GPU vendors as well as some FPGAs.

Usage

The main idea is very simple: You provide three pieces of information that will be used by the infrastructure to run benchmarks. These are:

• a recipe saying what the benchmark does
• a checker saying how to verify correctness of the benchmarks
• a logger saying how to record what happens during the benchmark

The main loop of the program is as simple as (up to technical details)

while (not recipeExhausted) {
  result = logger.call([]{recipe.next();});
  logger.call([]{checker.check(result);});
}

See examples for recipes inspirations and technical details.

Installation

Build and run a target

The project contains the following separate targets:

• test
• examples (and all subfolders)
• documentation
• all

All of those can be built by the following process:

cmake -S /path/to/<target> -B /path/to/build/<target> -Dalpaka_ACC_CPU_B_SEQ_T_SEQ_ENABLE:BOOL=ON
cmake --build /path/to/build/<target> [--target GenerateDocs]

wherein [--target GenerateDocs] is to be used only for building docs. This sets Dalpaka_ACC_CPU_B_SEQ_T_SEQ_ENABLE which allows running kernels sequentially on the host CPU. See the alpaka docs for details including other options.

Running the tests

Use the following commands from the project's root directory to run the test suite.

# build tests as above
CTEST_OUTPUT_ON_FAILURE=1 cmake --build /path/to/build/test --target test

# or simply call the executable:
/path/to/build/test/KitGenBenchTests

To collect code coverage information, run CMake with the -DENABLE_TEST_COVERAGE=1 option.

Build the documentation

The documentation is automatically built and published whenever a GitHub Release is created. To manually build documentation, call the following command.

cmake -S documentation -B build/doc
cmake --build build/doc --target GenerateDocs
# view the docs
open build/doc/doxygen/html/index.html

To build the documentation locally, you will need Doxygen, jinja2 and Pygments installed on your system.

Build everything at once

The project also includes an all directory that allows building all targets at the same time. This is useful during development, as it exposes all subprojects to your IDE and avoids redundant builds of the library.

cmake -S all -B build
cmake --build build

# run tests
./build/test/KitGenBenchTests
# run standalone
./build/examples/*/KitGenBenchExample* --help
# build docs
cmake --build build --target GenerateDocs

Additional tools

The test and standalone subprojects include the tools.cmake file which is used to import additional tools on-demand through CMake configuration arguments. The following are currently supported.

Sanitizers

Sanitizers can be enabled by configuring CMake with -DUSE_SANITIZER=<Address | Memory | MemoryWithOrigins | Undefined | Thread | Leak | 'Address;Undefined'>.

Static Analyzers

Static Analyzers can be enabled by setting -DUSE_STATIC_ANALYZER=<clang-tidy | iwyu | cppcheck>, or a combination of those in quotation marks, separated by semicolons. By default, analyzers will automatically find configuration files such as .clang-format. Additional arguments can be passed to the analyzers by setting the CLANG_TIDY_ARGS, IWYU_ARGS or CPPCHECK_ARGS variables.

Ccache

Ccache can be enabled by configuring with -DUSE_CCACHE=<ON | OFF>.