Snapshots on a schedule, replicas on another box: sanoid and syncoid pull replication for ZFS

A ZFS snapshot is a read-only picture of a dataset at one instant. It takes about a second to create and costs no space at first. Snapshots on a schedule, thinned automatically and copied to a second machine, are the single biggest upgrade most homelab storage can get. This post builds exactly that with two tools from the same project: sanoid takes and prunes snapshots, syncoid replicates them.

The trap

Homelab storage usually fails in one of two opposite ways. The first lab has no snapshots at all, so an accidental rm -rf on a share is permanent. The second takes snapshots every fifteen minutes and never prunes them: ten thousand of them, a pool that is mysteriously full, and a zfs list -t snapshot that takes a minute to answer.

Both labs also believe the snapshots are their backup. They are not. A snapshot lives in the same pool as the data. If the pool dies (two disks in a RAIDZ1, a controller that writes garbage, a mistaken zpool destroy), the snapshots die with it. Snapshots protect against changes: deletions, overwrites, bad upgrades, ransomware on a share. Backups protect against loss. You need both, and replication to a second box gets you most of the way.

Why snapshots are cheap

ZFS never overwrites data in place. When a block changes, the new version goes to a new location and the dataset's pointers move to it. A snapshot simply keeps the old pointers, so until something changes the snapshot and the live dataset share every block.

That also tells you where snapshot space goes. A snapshot grows as the live data moves away from it: delete a 20 GB folder and the space is not freed until every snapshot that still references those blocks is destroyed. When a pool fills up unexpectedly, this command sorts snapshots by the space only they hold, and the culprit is usually at the bottom:

zfs list -t snapshot -o name,used -s used

Step 1: a retention policy with sanoid

Debian and Proxmox VE package sanoid, and the package ships a systemd timer that runs it every 15 minutes:

apt install sanoid
systemctl enable --now sanoid.timer
systemctl list-timers | grep sanoid

The policy lives in /etc/sanoid/sanoid.conf: datasets at the top, templates at the bottom. A trimmed version:

[tank/documents]
	use_template = important
[tank/shares]
	use_template = standard
	recursive = yes
[tank/media]
	use_template = light

[template_important]
	frequently = 0
	hourly = 48
	daily = 30
	weekly = 8
	monthly = 12
	yearly = 0
	autosnap = yes
	autoprune = yes

[template_standard]
	hourly = 24
	daily = 14
	weekly = 4
	monthly = 3
	autosnap = yes
	autoprune = yes

[template_light]
	hourly = 0
	daily = 7
	weekly = 0
	monthly = 0
	autosnap = yes
	autoprune = yes

Each dataset section names a template, and recursive = yes applies it to every child dataset. The numbers are how many snapshots of each kind to keep. Recent history is dense (48 hourlies) and older history is sparse (12 monthlies), so about a hundred snapshots cover a full year, and sanoid deletes each one when it falls off the end of its rung.

Anything you leave out comes from /etc/sanoid/sanoid.defaults.conf, which you should not edit, so set every interval explicitly in your own templates.

Test the policy without touching anything, then take a round of snapshots and look:

sanoid --cron --readonly --verbose      # simulate
sanoid --take-snapshots --verbose       # take snapshots now
zfs list -t snapshot -o name,used -s creation tank/documents | tail -5

One rule if the box also runs Proxmox guests: pick one snapshot owner per dataset. Proxmox's own VM snapshots and storage replication create and delete snapshots on the same zvols. If Proxmox replication or vzdump with PBS already covers your guests, leave tank/vm out of sanoid, or keep its policy very light.

Step 2: how incremental replication works

zfs send turns a snapshot into a stream of bytes; zfs receive turns that stream back into a dataset on another pool. The first send is full. Every later send is incremental: only the blocks that changed between two snapshots travel. Both sides must keep the last common snapshot, or the next incremental has nothing to build on.

By hand it looks like this, with nas and backup as SSH host aliases with key-based login:

zfs snapshot tank/documents@rep-1
zfs send tank/documents@rep-1 | ssh backup zfs receive -u backup/documents
zfs snapshot tank/documents@rep-2
zfs send -i @rep-1 tank/documents@rep-2 | ssh backup zfs receive -u backup/documents

zfs receive -u does not mount the received dataset, which avoids mountpoint clashes on the target. Tracking the last common snapshot for every dataset, every night, is the part nobody keeps up with. syncoid does it for you: it finds the newest common snapshot, sends the incremental, resumes interrupted transfers when both sides support it, and cleans up its own snapshots.

Step 3: pull, do not push

A backup box that pulls from the NAS is safer than a NAS that pushes to the backup box. If the NAS is compromised, an attacker holding its SSH key can delete everything on a push target. With pull replication, the NAS has no credentials for the backup box at all.

On the backup box, a small script run by cron at 01:30:

cat > /usr/local/sbin/pull-replicas <<'EOF'
#!/bin/sh
S="/usr/sbin/syncoid --recursive --no-sync-snap --quiet"
$S nas:tank/photos    backup/photos
$S nas:tank/documents backup/documents
EOF
chmod +x /usr/local/sbin/pull-replicas
echo '30 1 * * * root /usr/local/sbin/pull-replicas' > /etc/cron.d/pull-replicas

By default syncoid creates its own short-lived syncoid_ snapshot for each run. --no-sync-snap tells it to use the snapshots sanoid already took, which keeps the target's history identical to the source. --recursive includes child datasets. For an encrypted dataset, add --sendoptions=w for a raw send: the blocks travel and are stored still encrypted, and the backup box never needs the key. Keep every passphrase somewhere that does not depend on the pool it unlocks.

The received datasets still need pruning. On the backup box, give them a template that never takes snapshots and only prunes what replication brings in:

[backup/photos]
	use_template = backup
	recursive = yes
[backup/documents]
	use_template = backup
	recursive = yes

[template_backup]
	autosnap = no
	autoprune = yes
	hourly = 48
	daily = 30
	weekly = 8
	monthly = 12

For tighter security, replicate as a non-root user with ZFS permission delegation: on the NAS, zfs allow -u replicator send,hold,snapshot tank, and run syncoid on the backup box with --no-privilege-elevation. The receiving user needs create,mount,receive on the target dataset. Test the delegation once by hand before relying on it.

A TrueNAS box works as the target too. It speaks standard ZFS over SSH, so the same syncoid commands work; create the destination parent dataset in the TrueNAS UI first and let TrueNAS own the retention for those datasets.

Step 4: prove it

Replication you have never restored from is a hope, not a plan. Once a month, restore one real file from the replica and compare checksums from both machines:

# On the backup box
zfs clone backup/documents@autosnap_2026-10-16_00:00:02_daily backup/restore-test
zfs set mountpoint=/mnt/restore-test backup/restore-test
sha256sum /mnt/restore-test/taxes/2025.pdf
ssh nas sha256sum /tank/documents/taxes/2025.pdf
zfs destroy backup/restore-test

Matching checksums are the proof.

Checklist

• Every dataset that matters has a sanoid template, and tank/vm has exactly one snapshot owner.
• sanoid.timer is enabled and zfs list -t snapshot shows recent autosnaps.
• A second box receives incremental replication, ideally by pulling.
• Encrypted datasets replicate raw, and the keys are stored off the pool.
• The backup box prunes with a backup template (autosnap = no).
• You have restored one file from the replica and compared checksums.

Found this useful? Subscribe on YouTube for the daily video, or follow the RSS feed.