custom_test_runners.md

Custom Test Runners

The bazel_integration_test macro supports executing tests with a custom test runner. So, if your integration tests require custom setup code or if you would prefer to write the integration tests in a specific language, you can create an executable target and pass it to the test_runner attribute.

Table of Contents

• Implementation of a Custom Test Runner
• Implementation of a Custom Test Runner that Modifies Source Files

Implementation of a Custom Test Runner

A custom test runner needs two pieces of information that can be discovered via environment variables:

1. Location of the Bazel binary (BIT_BAZEL_BINARY)
2. Location of the workspace directory under test (BIT_WORKSPACE_DIR)

The value for each environment variable is an absolute path.

The Bazel integration test framework expects a test runner to signal success by exiting with a zero exit code. A test runner that exits with a non-zero exit code will be considered a failed test.

Examples:

• Swift: test runner, usage
• Python: test runner, usage

Implementation of a Custom Test Runner that Modifies Source Files

If an integration test needs to modify source files that are under test, it is best to create a copy of the worksapce directory. By default, files provided to the integration test are symlinks to the actual source files. Modifications to these files will change the actual sources!

A utility, called create_scratch_dir.sh, provides a convenient way to create a copy of a workspace directory that can be safely modified by an integration test.

To use the utility, add @rules_bazel_integration_test//tools:create_scratch_dir as a dependency to your test runner binary target:

sh_binary(
    name = "use_create_scratch_dir_test_runner",
    testonly = True,
    srcs = ["use_create_scratch_dir_test.sh"],
    data = [
        "@rules_bazel_integration_test//tools:create_scratch_dir",
    ],
    deps = [
        "@bazel_tools//tools/bash/runfiles",
        "@cgrindel_bazel_starlib//shlib/lib:assertions",
    ],
)

In your test runner code, you will need to locate the utility. Bazel provides helper functions for locating dependencies in shell binaries.

create_scratch_dir_sh_location=rules_bazel_integration_test/tools/create_scratch_dir.sh
create_scratch_dir_sh="$(rlocation "${create_scratch_dir_sh_location}")" || \
  (echo >&2 "Failed to locate ${create_scratch_dir_sh_location}" && exit 1)

Create a copy of the workspace directory into a scratch directory using the utility.

scratch_dir="$("${create_scratch_dir_sh}" --workspace "${BIT_WORKSPACE_DIR}")"

Change into the scratch directory, make your file modifications, and execute your tests.

# Change into scratch directory
cd "${scratch_dir}"

# Modify the files in the scratch directroy
echo "Make a meaningful change." > foo.txt

# Execute tests in the scracth directory
"${BIT_BAZEL_BINARY}" test //...

Examples:

• A custom test runner that uses create_scratch_dir.sh