[Question] Can an entire resource be a valid binding value?

[jlucktay]

Describe your question

Hello, and thank you for Chainsaw! It's powerful and has loads of potential.
I've been starting to use it for some assertions, to make sure that our Crossplane resources pass muster.

Question: Is it possible/viable to bind an entire resource, for later reference in an assertion?

These are the relevant lines from my chainsaw-test.yaml file:

---
# yaml-language-server: $schema=https://github.com/kyverno/chainsaw/raw/main/.schemas/json/test-chainsaw-v1alpha1.json
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test

metadata:
  name: xrd

spec:
  steps:
    - name: Bind XRD to check against Compositions
      try:
        - assert:
            bindings:
              - name: xrd
                value:
                  resource:
                    apiVersion: apiextensions.crossplane.io/v1
                    kind: CompositeResourceDefinition
                    metadata:
                      name: crossplane-xrd
            resource:
              apiVersion: apiextensions.crossplane.io/v1
              kind: Composition
              metadata:
                name: crossplane-composition
              spec:
                compositeTypeRef:
                  apiVersion:
                    (ends_with(@, '/v1alpha1')): true
                    (ends_with(@, '/v1alpha2')): false
                    ## Uncommenting this assertion gives an error.
                    # (ends_with(@, $xrd.versions[?referenceable].name)): true

When I add xrd in the bindings list and give the value as a resource object, Chainsaw doesn't seem to have any issues with that syntax.
I could absolutely see this part being a gap in the implementation, though.

If I uncomment the last assertion on the bottom line, I get the below error:

spec.compositeTypeRef.apiVersion.(ends_with(@, $xrd.versions[?referenceable].name)): Internal error: invalid type for: <nil>, expected: []functions.JpType{"string"}

chainsaw version Version

v0.2.8

Additional context

I would like to be able to check the .spec.compositeTypeRef.apiVersion field of all relevant Composition-type resources.
I want the version on the end of the apiVersion string field to match the version from the XRD that has referenceable: true set.

There is #1034 about a lookup operation and I thought this could be similar, or maybe the existing get operation can cover off this use case, and wanted to ask for clarity's sake.
More generally, being able to bind (or lookup or get) one or more specific resources and refer to them in assertions against other resources would be fantastic.

[eddycharly]

Not sure I understand the question.
Any object can be assigned to a binding.

The $xrd binding above contains the object you put in the value field.

You can either use a script with an output or leverage x_k8s_get jp function.

[jlucktay]

Any object can be assigned to a binding.

OK, thank you for clarifying, I wasn't sure if this was valid or not.

The $xrd binding above contains the object you put in the value field.

Should I be able to assert directly against the binding?
In the code I posted above, on the error that comes from the commented out assertion line, it seems like it is expecting a list of JMESPath strings, but it finds nil.

You can either use a script with an output or leverage x_k8s_get jp function.

I could not find any docs or description or even code for the x_k8s_get function, searching on Google or here on GitHub, so I don't know which parameter is which in the signature on the Chainsaw functions page: x_k8s_get(any, string, string, string, string)

I will give output a try and see if I can get any further with that, but I still wonder if I'm doing anything wrong with that assertion line that is commented out.

So now the question has become: how do I assert against the values of an object that is assigned to a binding?

[eddycharly]

In the code I posted above, on the error that comes from the commented out assertion line, it seems like it is expecting a list of JMESPath strings, but it finds nil.

You are missing my point, $xrd.versions doesn't even exists, look at the value definition, you can use $xrd.resource.apiVersion for example (it's the object you defined, nothing more).

I could not find any docs or description or even code for the x_k8s_get function

Yeah, our docs for functions are not good, i would love to do better and should try to push that to the jp lib.
Here's an example of x_k8s_list https://kyverno.github.io/chainsaw/latest/examples/non-resource-assertions/?h=x_k8s, x_k8s_get should be pretty similar.

[eddycharly]

Unfortunately i have no experience with crossplane but let me know if i can help with the syntax.

[jlucktay]

You are missing my point, $xrd.versions doesn't even exists, look at the value definition, you can use $xrd.resource.apiVersion for example (it's the object you defined, nothing more).

Aha, I think this is the assumption that was tripping me up.
So using binding.resource in the test YAML doesn't function the same way that assert.resource in a test does, in how it looks up an existing object like the last example on this page.
That binding.resource is just hard-coded in there, WYSIWYG, whereas assert.resource has extra functionality and acts differently, retrieving an object that matches those parameters in the cluster.

On the binding side, this is where x_k8s_get would come into play, I guess? To actively retrieve the whole object into the value of the binding?
Something like:

spec:
  steps:
    - bindings:
        - name: boundResource
          value: (x_k8s_get(...))
      try:
        - description: Assert that two different resources are in the same namespace.
          assert:
            resource:
              apiVersion: something/v1
              kind: Something
              metadata:
                (namespace == $boundResource.namespace): true

I could not find any docs or description or even code for the x_k8s_get function

Yeah, our docs for functions are not good, i would love to do better and should try to push that to the jp lib. Here's an example of x_k8s_list https://kyverno.github.io/chainsaw/latest/examples/non-resource-assertions/?h=x_k8s, x_k8s_get should be pretty similar.

Thanks!

I will try and play with x_k8s_get a bit to see how I go.
Hopefully afterwards, I can contribute back with some extra docs on all of this to clarify.

[jlucktay]

I just created PR #1915 which adds an example using x_k8s_get() in a script and an assert.