Backups that actually survive a disaster
A backup you have never restored is a hope, not a plan. Notes on building recovery you can trust.
Most teams discover their backup strategy has a hole at the worst possible moment — during a real incident. After years of running production infrastructure, I’ve settled on a few non-negotiables.
The 3-2-1 rule, but honestly
The classic rule: 3 copies of your data, on 2 different media, with 1 off-site. It still holds up. The trap is treating it as a checkbox instead of a property you verify.
- 3 copies — the live data plus two backups, not the live data plus one backup you call “two” because it’s on RAID.
- 2 media — RAID is not a backup. Snapshots on the same array are not a backup. Different failure domains matter.
- 1 off-site — a fire, flood, or ransomware event that takes the rack should not take every copy.
Restores are the only test that counts
A backup job reporting success only proves bytes were written. It says nothing about whether you can get a working system back. So:
- Schedule periodic test restores to an isolated environment.
- Time them. Your recovery time objective (RTO) is fiction until you’ve measured a real restore.
- Verify the restored data, not just that the files exist.
If you have never restored it, you don’t have a backup — you have a hypothesis.
Protect the backups themselves
Ransomware now targets backup repositories first. Defenses that have earned their keep:
- Immutability / object lock so backups can’t be altered or deleted within a retention window.
- Separate credentials for the backup system — not the same domain admin that the rest of the fleet uses.
- Monitoring on backup health, with alerts when a job is missed, not just when one fails loudly.
The boring conclusion
Backup and disaster recovery isn’t exciting, and that’s the point. The goal is for a bad day to be boring — a measured, rehearsed recovery instead of an improvised scramble. Discipline up front buys calm later.