A backup job that reports TASK OK tells you that data was written. It does not tell you that the data is still readable, that you kept the right restore points, or that the copy would survive the event you are protecting against. Those are three separate questions, and Proxmox Backup Server answers each of them through a different mechanism.
This guide covers the three: how pruning and garbage collection actually interact, how the retention options combine, and how verification works. It ends with three traps that are documented, non-obvious, and expensive to discover during an incident.
Scope
- Proxmox Backup Server 4.x, with Proxmox VE clients. The mechanics described have been stable across the 3.x and 4.x series.
- Assumes PBS is installed and a datastore exists. Deploying it is a separate exercise.
- All commands here are read-only or clearly marked as changing configuration.
- Figures are starting points for a policy, not a policy. Yours depends on your recovery objectives.
Pruning and garbage collection are not the same operation
This is the single most common source of confusion, and it produces a recurring support question: "I pruned, why is my datastore not smaller?"
Pruning operates on snapshots. It applies the retention policy and removes the restore points you no longer want. What it removes is the snapshot and its index; the actual data chunks stay on disk, because other snapshots may still reference them.
Garbage collection operates on chunks. It runs in two phases. The mark phase reads every index file in the datastore and updates the access time of every chunk still referenced. The sweep phase walks every chunk on disk and compares its access time against a cutoff. Anything older than the cutoff is referenced by nothing and can be deleted.
The cutoff is the interesting part. It is not the current time. It is either the start time of the oldest running backup writer, or the start of the garbage collection run minus 24 hours and 5 minutes, whichever is more conservative.
That grace period is not arbitrary. Linux filesystems are typically mounted with relatime, which updates a file's access time at most once a day rather than on every read. Without the window, a chunk that is genuinely in use but whose access time was last batched more than a day ago could be mistaken for garbage and deleted while a backup depends on it. The 24-hour margin guarantees the mark phase's timestamp is reliably newer than the cutoff.
Three practical consequences follow.
Space is freed on the second garbage collection after a prune, not the first. Chunks inside the grace period are reported at the end of the task as pending removals. That line is not an error; it is the mechanism working.
Running garbage collection more than once a day buys you almost nothing, because nothing new becomes eligible inside the window. It is also expensive: it reads every index and touches every referenced chunk. Weekly is a reasonable default for most datastores, daily if your churn is high.
And anything else that touches the chunk directory resets the clock. A synchronisation tool walking the datastore at the filesystem level updates access times and can keep chunks alive indefinitely. If your datastore never shrinks and the schedule looks correct, look for a process reading the chunk store from outside PBS.
proxmox-backup-manager garbage-collection status <datastore>
Retention: the keep options combine, they do not chain
The retention options are keep-last, keep-hourly, keep-daily, keep-weekly, keep-monthly and keep-yearly. A snapshot survives if any rule wants to keep it. They are a union, not a sequence, which is why the total number of retained snapshots is usually smaller than the sum of the counters: one snapshot often satisfies several rules at once.
Two subtleties matter more than the arithmetic.
Periods without backups do not count. keep-daily=7 does not mean "seven days of history". It means "the latest snapshot from each of the last seven days that actually has one". If your job failed for four days, those days are skipped and the window silently reaches further back. This is usually what you want, and it is not what most people assume.
When several snapshots fall in the same period, only the latest is kept. Hourly backups with keep-daily=7 collapse to one per day beyond whatever keep-last and keep-hourly preserve.
A defensible starting policy for a production workload:
keep-last=3,keep-daily=14,keep-weekly=8,keep-monthly=12,keep-yearly=3
Read it as four different questions. The last three snapshots cover "I broke something an hour ago". Fourteen daily cover "the corruption started sometime last week". Eight weekly and twelve monthly cover "we noticed at the quarterly close". Three yearly exist because a regulator or a contract asked for them, and if nobody asked, remove the line rather than paying to store it.
Set the policy in one place. It can live on the Proxmox VE storage definition, on the backup job, or as a prune job on the PBS side. Defining it in two of those is how you end up with a retention that nobody can explain.
Verification: an unverified backup is an assumption
PBS protects chunk integrity at two levels. Each chunk carries a CRC-32 checksum that catches corruption at the block level, and each chunk is identified by its SHA-256 digest, which means any alteration invalidates the reference. On restore, integrity is checked before data is handed back.
That is protection at read time. Verification jobs are protection ahead of time: they read the chunks referenced by a snapshot and validate them, so you learn that a backup went bad months before you need it rather than during the incident.
A workable schedule treats recent and old backups differently. Verify new snapshots frequently, weekly or better, since that is where a bad disk shows up first. Re-verify older snapshots on a longer cycle, monthly or quarterly, and let the job skip snapshots already verified inside your chosen window so that the run stays proportionate.
Verification is read-intensive. Schedule it away from your backup window and away from garbage collection, which is also reading everything.
Trap 1: protection is not immutability
PBS lets you mark a snapshot as protected. Protected snapshots are excluded from automatic pruning and cannot be removed through the interface, the API or the CLI without explicitly unprotecting them first. It is the right tool for the backup you took before a major upgrade.
It is not immutability. The protection is enforced by PBS, and the backup files remain ordinary files on an ordinary filesystem. Anyone with root on the PBS host can delete them, and the protection flag does nothing about it.
That matters because of what backups are increasingly protecting against. An attacker who reaches your hypervisors and then your backup server does not need to defeat PBS; they need a shell. If your entire recovery plan lives on one PBS instance reachable with the same credentials as the rest of the estate, you have one copy in two places, not two copies.
The answers are architectural, not configuration flags. Distinct credentials for the backup server, with no shared authentication realm. A second PBS that pulls from the first rather than being pushed to, so that compromising the source does not grant write access to the target. Tape, where LTO WORM media is genuinely append-only. Or a copy that is physically offline for part of its life.
Decide which of these you have before you describe your backups as protected against ransomware.
Trap 2: lose the key, lose the backups
PBS encrypts client-side, with AES-256 in GCM mode. Data is encrypted before it leaves the Proxmox VE node, and the server stores only ciphertext. This is the correct design when the backup target is a machine you do not fully control: a hosted server, a colocation rack, a second site operated by someone else.
The consequence is absolute. The server has no key material. If you lose the encryption key, the data is unrecoverable, permanently, by anyone including Proxmox. There is no support ticket that fixes this.
So the key needs a lifecycle of its own. Store it somewhere that is not the PBS host and not the Proxmox VE nodes, because a disaster that takes out your infrastructure should not take the key with it. A secrets manager, a hardware security module, or a printed copy in a safe are all defensible; a file on the hypervisor is not. And test the recovery path from the stored copy, not from the copy that happens to be on the machine.
Trap 3: encryption keys fragment deduplication
This one is genuinely non-obvious, and it changes capacity planning.
Deduplication works by identifying chunks through their digest and storing each distinct chunk once. Virtual machine disks are split into fixed 4 MiB chunks; container filesystems are streamed into a pxar archive and split at content-defined boundaries by a rolling hash, which keeps chunk reuse high as files change. In an estate where many guests run similar operating systems, this is where the large deduplication ratios come from.
Encryption happens before the chunk reaches the server. Two identical blocks encrypted with two different keys produce two different chunks, with two different digests. They deduplicate against nothing.
The operational consequence: if you assign a separate encryption key per tenant, per department or per cluster, deduplication only ever operates inside each group. Fifty near-identical Debian guests split across five keys deduplicate as five sets of ten, not one set of fifty, and your datastore sizing is wrong by a factor you did not budget for.
That may still be the right call, because key separation is a real security boundary between tenants. The point is to make it a decision with a known cost rather than discovering the cost when the datastore fills.
The one test that matters
Everything above is preparation. The only evidence that a backup strategy works is a restore that worked.
Three levels are worth distinguishing. A structural check confirms the archive is readable and its index is valid, which verification jobs already do. A functional restore actually recreates the guest and confirms it boots and its services start. A deep restore validates application data: that the database is consistent, that the application answers, that the files users care about are there.
Only the first can be fully automated. Schedule the second monthly on a rotating sample, restoring to a new VMID on an isolated network so it cannot collide with production:
qmrestore <archive> 999 --storage local-lvm --unique
The --unique flag regenerates hardware identifiers including the MAC address. Without it, the restored guest is a duplicate of a machine that may still be running, on the same network, with the same address.
For single files, a full restore is usually the wrong tool. PBS can mount a snapshot's filesystem and let you pull individual paths:
proxmox-file-restore list <snapshot> /etc
proxmox-file-restore extract <snapshot> /etc/nginx/nginx.conf /tmp/nginx.conf
This is the operation your users will actually ask for, far more often than a disaster recovery, and it is worth having rehearsed it before someone is waiting.
A baseline schedule
| Task | Frequency | Notes |
|---|---|---|
| Backup | Daily, off-peak | Snapshot mode; suspend or stop only where consistency demands it |
| Prune | Daily, after the backup | Single policy definition, one place |
| Garbage collection | Weekly | More often buys nothing inside the grace window |
| Verify (recent) | Weekly | Away from the backup and GC windows |
| Verify (full) | Monthly or quarterly | Skip snapshots already verified |
| Sync to second site | Daily or continuous | Pull from the target, not push from the source |
| Functional restore test | Monthly, rotating sample | Isolated network, new VMID |
If you are configuring package repositories on the PBS or Proxmox VE hosts before any of this, the deb822 repository procedure applies to both products.
What this guide does not cover
Tape backup and LTO media pools, which are the serious answer to offline retention and deserve their own treatment. Object storage and cloud archive tiers, where retrieval latency changes what "recoverable" means. Synchronisation topologies in depth, including the direction of pull and the permission model that makes it a real boundary. And the installation of PBS itself, which is straightforward and separate.
Designing a retention and verification policy is not hard; keeping it true as the estate grows is, and that is an operations problem rather than a configuration one. We run backup and recovery as part of monitoring, maintenance and operational support, and review existing strategies under IT audit and architecture consulting. If you take one thing from this guide, make it the monthly restore test.