Bobay Stack

Ops & troubleshooting / 2026-08-16

When the clock drifts, certificates "expire" out of nowhere

Certificates and config are fine but the browser says insecure — check the clock first. A few minutes of drift is enough to break TLS validation.

3 MIN READ BY BOBAY.TOP Updated 2026-08-22 ↖ All notes
When the clock drifts, certificates "expire" out of nowhere

A machine that just came back from sleep, a VM that was suspended for a long time, a laptop that's been offline for weeks — any of them can have a skewed system clock. The symptom: visiting an HTTPS site reports an invalid certificate, while the certificate itself is perfectly fine.

Measure the drift first, then fix it

timedatectl status
chronyc tracking

The offset on the System time line is the key number. Tens of milliseconds is normal, seconds need attention, and past a few minutes problems are basically guaranteed.

Why virtual machines drift so easily

  • When the host hibernates/suspends, the guest clock doesn't advance
  • Under heavy load, clock interrupts get delayed and drift accumulates
  • Some cloud providers' time sources are unstable

In PVE you can install qemu-guest-agent in the VM and let the host sync time into it; containers share the host clock directly and need no extra handling.

A minimal chrony config

# /etc/chrony/chrony.conf
pool ntp.example.org iburst
makestep 1.0 3
rtcsync

makestep 1.0 3 means: during the first three syncs after startup, if the offset exceeds 1 second, step the clock directly instead of slewing it. Very useful for a machine that just resumed from suspend.

Troubleshooting order

  1. timedatectl — check the timezone and whether NTP is enabled
  2. chronyc sources -v — check whether any source is usable (^* marks the currently selected one)
  3. If neither works, check whether outbound UDP 123 is blocked
  4. On a cloud host, confirm the security group isn't blocking NTP

The lowest-effort fallback

Add a cron entry on critical machines to check the time once an hour and notify when the offset is large:

*/60 * * * * chronyc tracking | awk '/System time/ {print $4, $5}' >> /var/log/clock.log

TLS, Kerberos, two-factor authentication, log correlation — all of it depends on the clock. When this one thing breaks, symptoms pop up everywhere, but there is only one root cause.

Wrong time makes certificates "expired"

TLS validation uses the local machine's time. If the system clock is off by two years, the browser says the certificate has expired; if it's before the issuance date, it says the certificate is not yet valid. Both errors look like certificate problems but are actually clock problems.

The troubleshooting order should be: check date first, then the certificate.

date -u
timedatectl status
openssl x509 -in cert.pem -noout -dates

A minimal chrony config

# /etc/chrony/chrony.conf
pool ntp.aliyun.com iburst
pool time.cloudflare.com iburst
makestep 1.0 3
rtcsync

iburst makes it take samples quickly at startup, and makestep 1.0 3 allows the first 3 syncs to step the clock directly (step if the offset exceeds 1 second). Without makestep, a machine with a large offset spends hours "catching up slowly", and certificates keep failing the whole time.

Verify it actually synced

chronyc tracking      # check System time offset, Leap status
chronyc sources -v    # status of each source, ^* is the currently selected one

Leap status must be Normal. Not synchronised means not a single source connected — usually UDP 123 blocked by a firewall.

A window that's easy to overlook

If the certificate renewal job runs on a machine with an inaccurate clock, ACME's signature timestamp check fails and reports "bad nonce" or "JWS verification error", which has nothing to do with the certificate itself. Sync the clock first, then read the ACME logs — it saves half the troubleshooting time.

chronyNTPTLStime synchronization

References

chrony documentation ↗ RFC 5905 NTPv4 ↗