Autorestic with docker and chadburn

[mpfl]

I would like to start this discussion by thanking cupcakearmy and contributors for their hard work on autorestic, as well as the many many people who have worked on restic itself.

I like to make things difficult for myself* and so I like to run everything in docker containers. I'm also a big fan of services that monitor docker labels for configuration (see: traefik, diun, and flame).

To that end, I use PremoWeb/chadburn. It can monitor the docker daemon (via a docker socket proxy, of course) and run commands within containers or execute new containers in a similar fashion to cron.

[job-run "mace-backup"]
schedule = 0 2 * * *
image = cupcakearmy/autorestic:1.7.1
command = autorestic -c /data/.autorestic.yml backup -a
no-overlap = true
tty = true
volume = /home/autorestic/data/:/data:rw
volume = /home:/home:ro
volume = /redundant:/redundant:ro

The downside to using chadburn/docker containers is that each time a new container is created (which it is whenever chadburn runs the job configured above), a new, random, hexadecimal hostname is assigned to that container. As far as I can tell, there is no way to specify the hostname of a container within chadburn.

This means that autorestic forget considers each backup to be from a different host and therefore no pruning takes place.

Thankfully, we can restore this functionality with strategic application of autorestic exec, specifically to invoke forget --group-by paths,tags. Here is my sample config for chadburn:

[job-run "mace-backup-prune"]
schedule = 0 23 * * 0
image = cupcakearmy/autorestic:1.7.1
command = autorestic -c /data/.autorestic.yml exec -av -- forget --group-by paths,tags --keep-last 5 --keep-daily 7 --keep-weekly 1 --keep-monthly 12 --keep-yearly 10 --prune
no-overlap = true
tty = true
volume = /home/autorestic/data/:/data:rw

What you end up with is something like this:

    Executing on "mace-thyme-homes"

> Executing: /usr/bin/restic forget --tag ar:location:thyme-homes --group-by paths --keep-last 5 --keep-daily 7 --keep-weekly 1 --keep-monthly 12 --keep-yearly 10 --prune
Applying Policy: keep 5 latest, 7 daily, 1 weekly, 12 monthly, 10 yearly snapshots
keep 8 snapshots:
ID        Time                 Host          Tags                     Reasons           Paths
---------------------------------------------------------------------------------------------
a0bb992c  2022-07-15 17:00:04  94ecce2a447b  ar:location:thyme-homes  daily snapshot    /home
d7d5dea6  2022-07-16 17:00:04  81e2e6c7ed8c  ar:location:thyme-homes  daily snapshot    /home
a4256513  2022-07-17 17:00:04  0daeb45fdde4  ar:location:thyme-homes  daily snapshot    /home
7f50756a  2022-07-18 17:00:04  46895c4fde3b  ar:location:thyme-homes  last snapshot     /home
                                                                      daily snapshot
fb3ccc65  2022-07-19 17:00:04  472efb32c65d  ar:location:thyme-homes  last snapshot     /home
                                                                      daily snapshot
e9c249f9  2022-07-20 17:00:04  a9e360f3f2f3  ar:location:thyme-homes  last snapshot     /home
                                                                      daily snapshot
7a1c14ca  2022-07-26 02:41:40  83c82a34cc86  ar:location:thyme-homes  last snapshot     /home
de06d3bc  2022-07-26 17:00:03  d513a61ebba2  ar:location:thyme-homes  last snapshot     /home
                                                                      daily snapshot
                                                                      weekly snapshot
                                                                      monthly snapshot
                                                                      yearly snapshot
---------------------------------------------------------------------------------------------
8 snapshots

remove 6 snapshots:
ID        Time                 Host          Tags                     Paths
---------------------------------------------------------------------------
2c464f91  2022-07-09 02:43:04  5488fd06319b  ar:location:thyme-homes  /home
30e5fbf8  2022-07-09 03:12:54  e60e71ea55d0  ar:location:thyme-homes  /home
4e56418c  2022-07-11 00:34:41  c5447efaedc7  ar:location:thyme-homes  /home
17aaca60  2022-07-13 21:42:01  0b7eb0d8b4d6  ar:location:thyme-homes  /home
e0eb66e8  2022-07-14 03:34:33  07871b8a2945  ar:location:thyme-homes  /home
266c57af  2022-07-14 17:00:04  1fec9172d7fd  ar:location:thyme-homes  /home
---------------------------------------------------------------------------

A big big big caveat on this is that if you have multiple hosts backing up to the same backend with the same path and same location name (and therefore the same automatic tag from autorestic), you're gonna have a bad time. I have two hosts running autorestic and I make sure that host's name is somewhere in the location name e.g. mace-homes and thyme-homes. I even go as far as configuring a new backend for each location, but that might be overkill.

HTH, HAND

• Actually I like how easy docker containers make it to do a LOT of things. RIP dependency hell.