Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Run test conditionally #1160

Open
1 task done
karentu2 opened this issue Apr 15, 2024 · 2 comments
Open
1 task done

[Feature] Run test conditionally #1160

karentu2 opened this issue Apr 15, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@karentu2
Copy link

Problem Statement

For example, I want to run a chainsaw test if a Kyverno policy is in Enforce mode, but if the Kyverno policy is in Audit mode, then the chainsaw test is not run.

Solution Description

One way I can think of to implement this is to add some precondition check field under spec, in the example below I named it precheck. So if the precheck fails, then the test is skipped.

apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: example
spec:
  precheck:
  - try:
    - assert:
        file: path/to/kyverno-policy-enforce-mode.yaml # where this file indicates that a kyverno policy is in Enforce mode
  steps:
  - try:
    - apply:
        file: path/to/apply.yaml
    - assert:
        file: path/to/assert.yaml
    catch: []
    finally: []

Alternatives

It might be better to have the precheck field available on both the spec level so that the entire test can be skipped, and available on the spec.steps level so that specific steps can be skipped.

Additional Context

No response

Slack discussion

https://kubernetes.slack.com/archives/C067LUFL43U/p1712276658755649

Research

  • I have searched other issues in this repository and mine is not recorded.
@karentu2 karentu2 added the enhancement New feature or request label Apr 15, 2024
@mveitas
Copy link

mveitas commented May 21, 2024

I want to run a chainsaw test if a Kyverno policy is in Enforce mode, but if the Kyverno policy is in Audit mode, then the chainsaw test is not run

When we rollout a policy, we do this by our environment gradually (dev, test, and production) and control the value via helm values that are passed in. During our testing pipeline, we have added a script to ensure that Enforce mode is always set making our testing easier.

@IshwarKanse
Copy link

We would like to see this feature too. Some of our tests depend on the cluster env, cloud provider and auth types. For example, we are currently running these tests as below.

# Get the platform type
dt_platform_type=$(oc get infrastructures cluster -o=jsonpath='{.status.platformStatus.type}')
echo "Platform is $dt_platform_type"

# Check if the cluster is STS or WIF cluster
dt_wif_or_sts=$(oc get authentication cluster -o=jsonpath='{.spec.serviceAccountIssuer}')
echo "$dt_wif_or_sts"

if [[ "$dt_platform_type" == "AWS" && -n "$dt_wif_or_sts" ]]; then
    chainsaw test \
        --config .chainsaw-openshift.yaml \
        --report-name "junit_tempo_aws-sts" \
        --report-path "$ARTIFACT_DIR" \
        --selector type=aws-sts \
        --report-format "XML" \
        --test-dir tests/e2e-openshift-object-stores
else
    echo "Cluster is not AWS STS cluster, skipping the AWS STS tests"
fi

It would be great if this use case is handled in Chainsaw like if all the conditions match, proceed with the test run else skip the test case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants