stacks: pre/post script hooks

[lukasmrtvy]

Is your feature request related to a problem? Please describe.
In general, there isnt any way, how to handle secrets in Portainer, sops seems like an lightweigh option ( at render time ), how to keep versioned secrets in git near docker-compose files. Then, there isnt any way how handle external notifications, nothing is worse than automatic system without observability. Of couse, We can use exporters like node-exporter, cadvisor, etc, but We will still miss business smetrics like portainer_stacks_updated, portainer_stack_upgradable, etc. For now it should be possible to just trigger some webhook, for example with post hook. And at the same time prepare deployment with pre hook.

Describe the solution you'd like
Would be superb to be able to specify pre-hook and post-hook ( shell script, where can be internal variables referenced, like $PORTAINER_STACK_NAME, $PORTAINER_STACK_COMPOSE_PATH, $PORTAINER_STACK_STATUS, etc.. ) before and after stack deployment, especially with git automatically syncing stack from git repositry. This would allow us to:

• prepopulate env variables ( decrypt secrets sops -d secrets.enc.yaml --output-type dotenv > .env , not sure about injecting them into env in portainer )
• send notifications ( curl -X POST -H 'Content-type: application/json' --data '{"text":"Stack deployed.."}' https://slack...
• do additional stuff

Describe alternatives you've considered

• its already possible without Portainer with some CI

Additional context
Thanks

[huib-portainer]

Thanks for that suggestion, we'll take it into consideration.

[dot-mike]

As I'm in deep need of this solution my-self, I've been thinking about this issue - and I've realized it poses a security threat to allow blind execution of scripts pre/post-hook.

Note that to my needs I need to be able to decrypt some files on Git clone and that's what my thoughts are based around.

I first thought of a solution using Git Hooks, but the go-git library does not execute hooks nor is the global-config available for reading hooks defined globally in the system.

Then I was looking into a per repo yaml-language based file for defining hooks but that is just similar to a shell-script placed the git-repo, although a yaml-basd language file allows us to use pre-defined keys that Portainer can look for and validate to some extent. Using file defined in a repo limits the use-case to git-repositories only.

In the end I think a hook-tab in the stack page would be the solution as that would not only limit to Git-repos and this will allow us for controlling the user-input to some extend.

Regards to security, maybe sandbox the execution of arbitrary shell scripts in a docker container using Docker-in-Docker (dind)?

[dot-mike]

Attaching my workaround on this. I've created a patch for the 2.14 release supporting Git hooks by directly calling "git checkout" after the repository is cloned.
This hook is strictly limited to a static shell-script running on the Portainer filesystem, but that is good enough for me.

You can find the patch here: https://gist.github.com/dot-mike/d38427fd3776ca4a31de59e3e2fbf4e1

To make this hook work we use Git hooks as documented here: https://git-scm.com/docs/githooks
Add a shell script named post-checkout to the filesystem running Portainer and update the Git configuration pointing to the directory where post-checkout lives: git config --system core.hooksPath /etc/git/hooks

[sled]

I'm having the same need, but couldn't patch portainer because I intend to use the Business Edition.

I found it much simpler and cleaner to patch the compose-unpacker image, because Portainer allows to set the used image with an ENV variable called COMPOSE_UNPACKER_IMAGE.

My fork of compose-unpacker starts from the stack's docker compose file folder and traverses all sub-folders and decrypts all files containing ".sops." in their filename and writes them back without the ".sops." suffix. I.e. if you have a folder structure like:

docker-compose.yaml
general.env
secret.sops.env
subfolder/
  another.sops.env

it will produce:

docker-compose.yaml
general.env
secret.sops.env
secrets.env
subfolder/
  another.sops.env
  another.env

This allows your docker-compose file to reference them as env_file.

Github Repo of my fork: portainer/compose-unpacker@develop...sled:compose-unpacker:feature/automatic-sops-decryption
Github Docker image of my fork: https://github.com/users/sled/packages/container/package/compose-unpacker

How to use it

You need to set COMPOSE_UNPACKER_IMAGE to ghcr.io/sled/compose-unpacker:latest when starting portainer with the docker run command.

SOPS needs access to the key, you can set the key as ENV variable on your stack when you configure it in the UI, e.g. setting SOPS_AGE_KEY=AGE-SECRET-KEY-......

[dot-mike]

Interesting, thanks for sharing your workaround! Much appreciated.

Although, I must say I'm a bit disappointed coming back to this issue almost 2 years later to not see any progress!

[butenhoff-encoway]

@sled this looks very promising for my setup, will test later today.
Just to clarify: setup portainer with env: COMPOSE_UNPACKER_IMAGE: ghcr.io/sled/compose-unpacker:latest
and when deploying new git-stacks manually add env SOPS_AGE_KEY via UI and the stack uses my .env file as well as the sops key?