[bitnami/spring-cloud-dataflow] deployer.imagePullPolicy configuration not working if skipper is disabled

[kheckelmann]

Name and Version

bitnami/spring-cloud-dataflow

What architecture are you using?

amd64

What steps will reproduce the bug?

1. Configure the spring-cloud-dataflow helm chart with

   skipper:
      enabled: false
   deployer:
     imagePullPolicy: Always 
   server:
     configuration:
       batchEnabled: true
       streamingEnabled: false

2. Register a docker task application with a snapshot image
3. Run the task once
4. Overwrite the snapshot image of the task application by pushing a new image to the container registry
5. Run the task again
6. Observe if the new image is used for the task

What is the expected behavior?

I would assume that the deployer.imagePullPolicy set to Always would result in the new deployed task having the new snapshot image.

What do you see instead?

It is still the old image, I can see it if I inspect the container and a kubectl describe on the pod created for the task results in something like this:

Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  26s   default-scheduler  Successfully assigned XX/rt-1727717065483-t-0-1-o2ll60vegl to XXX
  Normal  Pulled     25s   kubelet            Container image "XXX:1.0.0-SNAPSHOT" already present on machine
  Normal  Created    25s   kubelet            Created container rt-1727717065483-t-0-1-7dkk27m4p4
  Normal  Started    25s   kubelet            Started container rt-1727717065483-t-0-1-7dkk27m4p4

So apparently the snapshot image does not get pulled.

Additional information

I had a look at the config map for the scdf server (https://github.com/bitnami/charts/blob/main/bitnami/spring-cloud-dataflow/templates/server/configmap.yaml) and it looks like the deployer.imagePullPolicy is not evaluated there. However, other properties like deployer.imagePullSecrets are evaluated and according to the spring-cloud-dataflow documentation (https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-kubernetes-deployer) the imagePullPolicy can be defined the same way. Therefore it looks like in general, the deployer configuration properties are meant to be used without a skipper - that's why I think this is a bug.

Since it was not in the template but I didn't want to overwrite the complete config map, I tried setting it as an environment variable like this:

server:
  extraEnvVars:
        - name: SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_IMAGEPULLPOLICY
          value: Always

With the environment variable, a kubectl describe on the pod created for the task results in:

Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  7m21s  default-scheduler  Successfully assigned XXX/rt-1727715294067-t-0-1-xpllze8zmd to XXX
  Normal  Pulling    7m21s  kubelet            Pulling image "XXX:1.0.0-SNAPSHOT"
  Normal  Pulled     7m20s  kubelet            Successfully pulled image "XXX" in 954.204174ms (954.212059ms including waiting)
  Normal  Created    7m20s  kubelet            Created container rt-1727715294067-t-0-1-d1ddl3ylqr
  Normal  Started    7m20s  kubelet            Started container rt-1727715294067-t-0-1-d1ddl3ylqr

So that worked for me, but if it is not left out intentionally for some reason, I would prefer to be able to use deployer.imagePullPolicy. I haven't found anything in the documentation that implies that this doesn't work by design, but I might have missed something. If that's the case, please point me to the correct hints, thank you.

[dgomezleon]

Hi @kheckelmann ,

Thanks for the heads-up. Since you did a deep investigation and are familiar with the issue, would you like to contribute with a PR to update the CM? The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.