{
    "version": "https://jsonfeed.org/version/1.1",
    "title": "Backup on jtagcat's tail",
    "home_page_url": "https://tail.jtag.cat/tags/backup/",
    "feed_url": "https://tail.jtag.cat/tags/backup/feed.json",
    "items": [
        {
            "id": "https://tail.jtag.cat/zfs-freeze/",
            "url": "https://tail.jtag.cat/zfs-freeze/",
            "date_published": "2023-10-03T16:18:19Z",
            "title": "zfs-freeze: External atomic backups",
            "content_html": "\u003cp\u003eI use \u003ca href=\"https://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html\"\u003e\u003ccode\u003ezfs-send\u003c/code\u003e\u003c/a\u003e for backups, but not all destinations\u003csup id=\"fnref:1\"\u003e\u003ca href=\"#fn:1\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e1\u003c/a\u003e\u003c/sup\u003e support it. Backing up live systems is problematic — while non-atomic \u003cstrong\u003eordered\u003c/strong\u003e rollbacks are business as usual, segmented backups lead to corrupted databases and broken restores.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://12factor.net\"\u003eTwelve-Factor Apps\u003c/a\u003e included: even if state isn\u0026rsquo;t derived from multiple sources, references can still break. For example, when blob storage is backed up a minute before the database, the restored application could hold a broken reference to an image, which doesn\u0026rsquo;t exist.\u003c/p\u003e\n\u003cp\u003eIn small (yet sizable) systems, surgical strategizing is unnecessary. With the power of \u003ccode\u003e$ zfs snapshot tank/apps/{db,s3}/cutie\u003c/code\u003e\u003csup id=\"fnref:2\"\u003e\u003ca href=\"#fn:2\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e2\u003c/a\u003e\u003c/sup\u003e and \u003ccode\u003ezfs-send\u003c/code\u003e, we are done. But what\u0026rsquo;s the shiny fridge all about?\u003c/p\u003e\n\u003cp\u003eStaying in the ecosystem, we trust ZFS with all copies of our data. To manage the risk, I backup to a technologically isolated system. However, generic tooling doesn\u0026rsquo;t provide the same consistency guarantees; thus I created a script to expose latest snapshots with multiple datasets.\u003c/p\u003e\n\u003chr\u003e\n\u003cp\u003eThe bash prototype wasn\u0026rsquo;t compatible with \u003ca href=\"https://en.wikipedia.org/wiki/Setuid\"\u003esetuid\u003c/a\u003e\u003csup id=\"fnref:3\"\u003e\u003ca href=\"#fn:3\" class=\"footnote-ref\" role=\"doc-noteref\"\u003e3\u003c/a\u003e\u003c/sup\u003e. Satisfying the binary requirement, I reimplemented it in Go, with extra features.\u003c/p\u003e\n\u003cp\u003eIllustrative usage of \u003ca href=\"https://github.com/jtagcat/jtagcat/tree/main/compile-scripts/zfs-freeze\"\u003e\u003ccode\u003ezfs-freeze\u003c/code\u003e\u003c/a\u003e:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sh\" data-lang=\"sh\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Dataset I want to backup: tank/top (and all sub-datasets)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# One-time setup\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ego install github.com/jtagcat/jtagcat/compile-scripts/zfs-freeze@latest\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003esudo chown root:root \u003cspan class=\"nv\"\u003e$GOPATH\u003c/span\u003e/bin/zfs-freeze\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003esudo chmod \u003cspan class=\"m\"\u003e4755\u003c/span\u003e \u003cspan class=\"nv\"\u003e$GOPATH\u003c/span\u003e/bin/zfs-freeze\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003esudo mv \u003cspan class=\"nv\"\u003e$GOPATH\u003c/span\u003e/bin/zfs-freeze /usr/local/bin\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ezfs create tank/_freeze_top\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e# Regular backups:\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003efrozen_dataset\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"k\"\u003e$(\u003c/span\u003ezfs-freeze tank/top restic-cup-\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"k\"\u003e$(\u003c/span\u003edate --rfc-3339\u003cspan class=\"o\"\u003e=\u003c/span\u003edate\u003cspan class=\"k\"\u003e)\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"k\"\u003e)\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e \u003cspan class=\"c1\"\u003e# will create tank/_freeze_top/restic-cup-2023-10-03\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003efunction\u003c/span\u003e cleanup \u003cspan class=\"o\"\u003e{\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  zfs-freeze -d \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$frozen_dataset\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"o\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nb\"\u003etrap\u003c/span\u003e cleanup EXIT\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003erestic -r cup: -p crypt.key backup \u003cspan class=\"s2\"\u003e\u0026#34;/\u003c/span\u003e\u003cspan class=\"nv\"\u003e$frozen_dataset\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdetails\u003e\u003csummary\u003eTemplates for Hetzner: rsync and \u003ca href=\"https://restic.net/\"\u003erestic\u003c/a\u003e\u003c/summary\u003e\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sh\" data-lang=\"sh\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nv\"\u003efrozen_dataset\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"k\"\u003e$(\u003c/span\u003ezfs-freeze tank/top restic-cup-\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"k\"\u003e$(\u003c/span\u003edate --rfc-3339\u003cspan class=\"o\"\u003e=\u003c/span\u003edate\u003cspan class=\"k\"\u003e)\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"k\"\u003e)\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"k\"\u003efunction\u003c/span\u003e cleanup \u003cspan class=\"o\"\u003e{\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  zfs-freeze -d \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$frozen_dataset\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"o\"\u003e}\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nb\"\u003etrap\u003c/span\u003e cleanup EXIT\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sh\" data-lang=\"sh\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003ersync -e \u003cspan class=\"s2\"\u003e\u0026#34;ssh -p23 -oUserKnownHostsFile=\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HOME\u003c/span\u003e\u003cspan class=\"s2\"\u003e/.ssh/known_hosts -oIdentityFile=hetzner_\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e.key\u0026#34;\u003c/span\u003e \u003cspan class=\"se\"\u003e\\\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"se\"\u003e\u003c/span\u003e      --archive --delete --compress \u003cspan class=\"se\"\u003e\\\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"se\"\u003e\u003c/span\u003e      --exclude\u003cspan class=\"o\"\u003e=\u003c/span\u003e.stversions \u003cspan class=\"se\"\u003e\\\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"se\"\u003e\u003c/span\u003e      --relative -- /\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$frozen_dataset\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e/./\u003cspan class=\"o\"\u003e{\u003c/span\u003edirs,to,backup\u003cspan class=\"o\"\u003e}\u003c/span\u003e/ \u003cspan class=\"se\"\u003e\\\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"se\"\u003e\u003c/span\u003e      \u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e@\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e.your-storagebox.de:\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-sh\" data-lang=\"sh\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003erestic -r sftp:: -p restic.key -o sftp.command\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;ssh \u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e.your-storagebox.de -p23 -oIdentityFile=hetzner_\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e.key -l \u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e -s sftp\u0026#34;\u003c/span\u003e unlock --quiet\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003erestic -r sftp:: -p restic.key -o sftp.command\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;ssh \u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e.your-storagebox.de -p23 -oIdentityFile=hetzner_\u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e.key -l \u003c/span\u003e\u003cspan class=\"nv\"\u003e$HETZNER_USER\u003c/span\u003e\u003cspan class=\"s2\"\u003e -s sftp\u0026#34;\u003c/span\u003e backup --quiet --exclude-file \u003cspan class=\"s2\"\u003e\u0026#34;restic.excludes\u0026#34;\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;/\u003c/span\u003e\u003cspan class=\"nv\"\u003e$frozen_dataset\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003c/details\u003e\n\n\u003cp\u003eWriting the helper, I yearned to use \u003ca href=\"https://github.com/jtagcat/hcli\"\u003ehcli\u003c/a\u003e: I could\u0026rsquo;ve avoided errors in parsing arguments, and skipped a refactor.\u003c/p\u003e\n\u003cdiv class=\"footnotes\" role=\"doc-endnotes\"\u003e\n\u003chr\u003e\n\u003col\u003e\n\u003cli id=\"fn:1\"\u003e\n\u003cp\u003eExhibit A: Hetzner\u0026rsquo;s (cheap!) pure ZFS storage boxes\u0026#160;\u003ca href=\"#fnref:1\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:2\"\u003e\n\u003cp\u003eHelpers such as \u003ca href=\"https://github.com/jimsalterjrs/sanoid\"\u003eSanoid\u003c/a\u003e and \u003ca href=\"https://github.com/zrepl/zrepl\"\u003ezrepl\u003c/a\u003e, do not snapshot \u003cstrong\u003eacross datasets\u003c/strong\u003e atomically, only in rapid succession (\u003ccode\u003e$ zfs snapshot tank/apps/db/cutie \u0026amp;\u0026amp; zfs snapshot tank/apps/s3/cutie\u003c/code\u003e).\u0026#160;\u003ca href=\"#fnref:2\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli id=\"fn:3\"\u003e\n\u003cp\u003eOutside of \u003ca href=\"https://nixos.org/\"\u003eNix\u003c/a\u003e, setuid wrappers are a hassle.\u0026#160;\u003ca href=\"#fnref:3\" class=\"footnote-backref\" role=\"doc-backlink\"\u003e\u0026#x21a9;\u0026#xfe0e;\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003c/div\u003e\n"
        }
    ]
    
}