zfs-freeze: Backup without zfs-send

• 2 minute read • backuplinux

I use zfs-send for backups, but most destinations1 don’t support it. Backing up a live system is problematic, since the files change. This means corrupted databases and broken restores for non-twelve-factor apps: state is derived from multiple sources, which are backed up at different times.

The idea is to back up off of ZFS snapshots. The datasets contain sub-datasets, which all have their own snapshots. Initially, I implemented a helper in bash, but discovered that using setuid is not possible for non-binary scripts.

I didn’t want to run my backup jobs as root. Outside of Nix, setuid wrappers are a hassle. Satisfying the binary requirement, I reimplemented it in Go as zfs-freeze.2

Illustrative usage:

# Dataset I want to backup: tank/top

# One-time setup
go install github.com/jtagcat/jtagcat/compile-scripts/zfs-freeze@latest
sudo chown root:root $GOPATH/bin/zfs-freeze
sudo chmod 4755 $GOPATH/bin/zfs-freeze
sudo mv $GOPATH/bin/zfs-freeze /usr/local/bin

zfs create tank/_freeze_top

# Regular backups:
frozen_dataset="$(zfs-freeze tank/top restic-cup-"$(date --rfc-3339=date)")" # will create tank/_freeze_top/restic-cup-2023-10-03
restic -r cup: -p crypt.key backup "/$frozen_dataset"
zfs-freeze -d "$frozen_dataset" # cleanup
Actual rsync and restic commands for Hetzner (simplified)
frozen_dataset="$(zfs-freeze tank/top restic-cup-"$(date --rfc-3339=date)")"
function cleanup {
  zfs-freeze -d "$frozen_dataset"
trap cleanup EXIT
rsync -e "ssh -p23 -oUserKnownHostsFile=$HOME/.ssh/known_hosts -oIdentityFile=hetzner_$HETZNER_USER.key" \
      --archive --delete --compress \
      --exclude=.stversions \
      --relative -- /$frozen_dataset/./{dirs,to,backup}/ \
restic -r sftp:: -p restic.key -o sftp.command="ssh $HETZNER_USER.your-storagebox.de -p23 -oIdentityFile=hetzner_$HETZNER_USER.key -l $HETZNER_USER -s sftp" unlock --quiet
restic -r sftp:: -p restic.key -o sftp.command="ssh $HETZNER_USER.your-storagebox.de -p23 -oIdentityFile=hetzner_$HETZNER_USER.key -l $HETZNER_USER -s sftp" backup --quiet --exclude-file "restic.excludes" "/$frozen_dataset"

Writing zfs-freeze, I really wanted for hcli to be ready. While developing, I made errors parsing arguments. Before refactoring, the code looked cluttered.

  1. Exhibit A: Hetzner’s pure ZFS storage boxes ↩︎

  2. Of course, with scope-creep. ↩︎