Year-End Server Maintenance: Patching, Snapshots and Log Rotation
It’s the second week of December, the office is half-empty, and you’re staring at a change window that nobody else wants. The ticketing system has gone quiet, the only Slack messages are about the Christmas do, and you’ve got a terminal open on a FreeBSD box that’s been humming along since March. This is the moment we do our year-end maintenance pass — not because of tradition, and definitely not because of superstition, but because a few disciplined hours in December buy us a calm January. Every sysadmin we know has a horror story about a patch applied in the first week of the new year, with everyone back at their desks and every service under load. We’d rather do the work now, while business traffic is low and the phone isn’t ringing.
Why We Treat Year-End Maintenance as a Discipline, Not a Ritual

There’s a temptation to treat December maintenance as a box-ticking exercise — reboot a few servers, run freebsd-update, call it done. That’s not what we’re doing here. We’re talking about a structured pass across the whole estate: patching, snapshotting, log rotation, jail checks, and a written record of what changed. It’s boring work, and that’s the point.
The UK Holiday Change Freeze and Why It Matters
Most UK organisations run some form of winter holiday change freeze, typically from mid-December through the first working week of January. The reasoning is sound: staff are on leave, on-call rotas are thin, and a failed change on Christmas Eve means waking someone up at 2am who’d rather be asleep. But the freeze also creates an opportunity. In the days before it kicks in, you have low traffic, quiet monitoring, and a genuine window to do work that would be risky in a busy period. We plan our maintenance pass for the week before the freeze, not during it.
What We Actually Want to Achieve Before 1 January
By the time we clock off for the holidays, we want four things to be true:
- Every host is patched to a supported security level, with reboots completed and validated.
- Fresh ZFS snapshots exist on every dataset we care about, with a known retention policy.
- Log rotation is tuned so nothing fills
/varwhile we’re eating mince pies. - Jails, services, and internal mirrors are checked and documented.
None of this is glamorous. All of it prevents the January firefighting that eats the first fortnight of the year.
Patching FreeBSD: Base System, Ports and Packages
Our patching order is deliberate: base system first, then packages, then ports if we’re managing any from source. We read the security advisories before we touch anything, because a patch that fixes one thing can occasionally break another, and we want to know what we’re walking into.
Freebsd-update and the Base System
The base system is handled with freebsd-update. On a typical 14.x host, that means:
freebsd-update fetchto pull the latest patches.freebsd-update installto apply them.- A reboot if the kernel or critical userland binaries changed.
- A second
freebsd-update installafter reboot to finish userland updates.
On FreeBSD 13.x hosts, the process is identical — but we check the branch’s support status first. If a box is still on 13.1 or 13.2, this is the moment to plan an upgrade to a supported minor release rather than just patching and hoping. We also confirm that freebsd-version -kru reports consistent kernel, userland, and running versions after the reboot. If those three don’t match, something didn’t land properly.
Packages, Ports and pkg upgrade
Packages come next with pkg upgrade. We always run pkg update first, then review the list of changes before confirming — particularly on hosts running databases or web stacks where a minor version bump can change behaviour. For hosts where we build from ports, we refresh the tree with git pull in /usr/ports (or portsnap on older setups) and use portmaster or poudriere to rebuild. Poudriere is our preference on anything with more than a handful of custom options, because it gives us clean, reproducible packages and a build log we can actually read.
Reboots, Kernel Changes and Scheduling Downtime
Kernel changes require a reboot, and reboots require downtime. We schedule them explicitly, notify stakeholders, and stagger them across the estate so we’re never rebooting both nodes of a pair at once. For jails, the host reboot takes the jails with it — so we plan jail restarts as part of the same window rather than treating them separately. A short maintenance notice on the internal status page goes a long way towards avoiding awkward questions in January.
ZFS Snapshots: Your Safety Net Before You Touch Anything
Before we patch, upgrade, or edit a single config file, we snapshot. This is non-negotiable. ZFS snapshots are cheap, instant, and they’ve saved us more times than we can count — whether from a bad package upgrade, a botched pf.conf edit, or a config change that looked fine in testing and broke in production.
Recursive Snapshots and Naming Conventions
We snapshot recursively at the pool or parent dataset level so nothing gets missed:
zfs snapshot -r tank@pre-maint-2024-12
The naming convention matters. We use a consistent prefix (pre-maint-) followed by a date, so we can find and prune snapshots later without guessing. On hosts with multiple pools, we script the snapshot step so it runs across all of them in one pass. The whole operation takes seconds and costs almost nothing in space until data changes.
Boot Environments with bectl
On hosts where we’re doing a significant upgrade — a major FreeBSD version bump, for example — we go further and create a boot environment with bectl. A boot environment is a clone of the root dataset that you can boot into independently, which means if an upgrade goes sideways you can reboot straight back into the previous environment and carry on. We create one with bectl create pre-upgrade, activate it with bectl activate, and note the environment name in the change record. It’s the closest thing FreeBSD has to a “undo” button for a system upgrade.
Retention, Space and Cleaning Up Old Snapshots
Snapshots are only useful if you can afford to keep them. We set a retention policy — typically 30 days for routine maintenance snapshots, longer for pre-upgrade ones — and prune with zfs destroy on a schedule. We also check zfs list -t snapshot for anything unexpectedly large, because a snapshot holding onto a deleted multi-gigabyte database file is a classic way to fill a pool without noticing.
Log Rotation and Disk Hygiene on FreeBSD
Nothing ruins a quiet holiday quite like a full /var partition at 3am on Boxing Day. Log rotation is the unglamorous work that keeps that from happening.
Tuning newsyslog.conf for Busy Servers
FreeBSD’s newsyslog.conf handles rotation by default, and it’s usually fine for a lightly-loaded box. On busy servers — web frontends, mail relays, anything with chatty daemons — we tune it. The main things we adjust:
- Rotation size thresholds, so logs rotate on size rather than waiting for the daily cron.
- Compression (
Zflag) on rotated logs, which cuts disk usage dramatically. - Retention counts, so we’re not keeping 52 weeks of
auth.log. - Drop-in files in
/etc/newsyslog.conf.d/for service-specific rules, keeping the main config clean.
The drop-in directory is particularly useful on hosts running multiple roles, because each service’s rotation rules live in their own file and can be reviewed independently.
Compression, Retention and Remote Shipping
We compress rotated logs by default and ship anything security-relevant — auth logs, sudo logs, firewall logs — to a central collector. That gives us retention beyond what the local disk allows and means we’re not relying on a single host’s /var for audit trails. For UK organisations with data residency requirements, the collector lives in the same jurisdiction as the source hosts.
Watching /var and ZFS Dataset Growth
Before the holidays, we run df -h and zfs list across every host and record the numbers. Anything above 80% gets attention now rather than in January. We also check dataset growth trends — a dataset that’s grown 40% in a month is telling you something, and it’s better to find out while you’ve got time to act.
BSD Jails and Services: Checking the Whole Estate
Jails are one of FreeBSD’s best features and one of the easiest things to forget about during maintenance. A host can be fully patched while its jails are months behind.
Patching Jails Without Breaking the Host
We patch jails separately from the host, using ezjail or iocage depending on what the estate uses. With iocage, that’s typically iocage update per jail, followed by a restart. With ezjail, we use ezjail-admin update to refresh the basejail. Either way, we snapshot the jail’s dataset first, and we stagger restarts so dependent services don’t all bounce at once.
Service Checks and Port Audits
After patching, we verify what’s actually running. service -e lists enabled services, and sockstat -l shows what’s listening. We compare against our documented baseline — anything unexpected listening on a public interface gets investigated immediately. We also run a port audit to catch any packages with known vulnerabilities that weren’t covered by the upgrade.
Mirrors, Package Caches and UK Connectivity
If you run an internal FreeBSD mirror or package cache, this is the time to verify it. We check that the mirror is reachable, that its content is current, and that UK-based hosts are pulling from it rather than a distant upstream. A stale internal mirror is worse than no mirror at all, because it silently serves old packages to everything behind it. We also confirm that our pkg repositories point at the right sources and that any local poudriere repositories have been refreshed.
Documentation, Monitoring and a Calm January
The technical work is only half the job. If we don’t write down what we did, the next year-end pass starts from scratch.
Runbooks and Change Records
Every change goes in the change record with a timestamp, the host affected, the commands run, and the outcome. We update the runbook as we go — if a step was different from what the runbook said, the runbook is wrong and needs fixing. By the time January arrives, anyone on the team should be able to pick up the runbook and understand exactly what state the estate is in.
Monitoring and Alert Tuning
We check that monitoring is actually watching the right things. In Zabbix or Prometheus, that means confirming that disk, memory, and service checks are firing correctly, and that alerts route to the right on-call rota over the holiday period. We also take the opportunity to tune out noisy alerts that have been generating false positives for months — a quiet monitoring system is one you’ll actually trust when it does go off.
A Short Post-Maintenance Review
Once the work is done, we spend thirty minutes as a team reviewing what went well and what didn’t. Did a patch take longer than expected? Did a snapshot restore get tested? Did anything surprise us? We write it down. The point isn’t to assign blame — it’s to make next December’s pass faster and less stressful than this one.
FAQ
How often should we patch FreeBSD servers?
We patch on a monthly cadence as a baseline, with out-of-band patches for critical security advisories. The year-end pass is a deeper, more thorough version of the same routine — not a replacement for regular patching. If you’re only patching once a year, the December window will be much heavier than it needs to be.
Are ZFS snapshots a substitute for backups?
No. ZFS snapshots protect against accidental deletion, bad config changes, and failed upgrades on the same pool. They do not protect against pool corruption, disk failure, or a host being lost entirely. We treat snapshots as a safety net and keep proper off-host backups alongside them.
Should we patch jails before or after the host?
Host first, then jails. The host’s kernel and userland underpin the jails, so patching the host first avoids version mismatches. Snapshot each jail’s dataset before updating it, and restart jails in a staggered order to avoid a thundering herd of dependent services.
How do we know if our internal FreeBSD mirror is healthy?
Check that it responds, that its package index is current, and that a test host can install a package from it without falling back to an upstream source. We also monitor mirror disk usage and sync logs, because a mirror that’s silently stopped syncing is worse than no mirror at all.
What’s the single most useful thing to do before the holiday change freeze?
Take snapshots. Everything else — patching, log tuning, jail updates — can be recovered from if you have a recent snapshot to roll back to. Without one, a bad change in December becomes a very long night.
A disciplined year-end routine — patching the base system and packages, taking ZFS snapshots and boot environments before you touch anything, tuning newsyslog.conf, checking jails and mirrors, and writing it all down — turns January into a quiet month rather than a firefighting one. The work is unglamorous, but it’s the difference between starting the year with confidence and starting it with a pager going off at midnight.
