Skip to main content

Keeping Renewal Boring

Short-lived certificates are only a good idea if renewal is reliably automated. The price of short lifetimes is that expiry must be a non-event, and the only way to get there is automation.

Certificates with thie setup presented live 90 days at most, usually less. Across a dozen hosts, each with a server certificate and one or more client certificates, manual renewal is tedious, and risks that something will expire unnoticed, and take a service down. The goal is a system where certificates are renewed before expiry, and the only time a human is involved is when renewal fails.

Per-host renewal

Each host runs a renewal service that watches its own certificates and renews them as they approach expiry. As a systemd unit:

[Unit]
Description=Auto-renew internal host certificate
After=network.target

[Service]
ExecStart=/usr/bin/step ca renew \
  /etc/ssl/certs/service.crt \
  /etc/ssl/private/service.key \
  --daemon
Restart=always
RestartSec=10
User=root
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=step-renew

[Install]
WantedBy=multi-user.target

The important pieces:

  • --daemon: step runs continuously, waking on its own schedule to check whether the certificate is close enough to expiry to warrant renewal. It renews in place, so the certificate and key files are refreshed without intervention.
  • Restart=always: if the renewal daemon dies, systemd brings it back. A renewal daemon that has quietly stopped is the silent failure this whole system exists to avoid.
  • SyslogIdentifier=step-renew: renewal events are tagged in syslog. That tag is what makes renewal auditable so the logs can forwarded and can be reviewed with alerts triggered on failure.

The WAF's client certificates

The WAF holds a client certificate for every backend it talks to (waf-to-service, one per host). These are renewed on a weekly timer that reissues anything approaching expiry:

  • A service performs the renewal pass, reissuing any waf-to-<host> certificate that expires within the next eight days.
  • A timer runs it weekly.

The specific window matters. Renewing anything that expires within the next eight days on a weekly schedule means every certificate gets two renewal attempts before it lapses. If one weekly run is missed for some reason, the next still catches the certificate with one day to spare. This overlap is deliberate; you never want renewal timing to be so tight that a single missed run causes an outage.

Back to top