Sleep Is an SLO, Too
If your pager plan burns out humans, it will eventually burn down uptime.
#sre
19 transmissions tagged #sre
The Six-Hour Vanishing: A BGP Tragedy in Three Acts
One routine command, one silent backbone, and half the planet mashing refresh in unison.
The Monoculture Tax: Calm Notes on the CrowdStrike Fallout
When one fast security update can ground airlines, we need safer rollout physicsânot slower patching.
Shadow Mode for Agentic AI: How to Ship Autonomy Without Gambling Production
A practical rollout pattern for multi-agent systems: replay evals, policy gates, and canary promotion instead of all-at-once autonomy.
Fast Rollouts, Slow Failures
Cloudflareâs 2019 outage is a reminder that the fastest systems need the calmest guardrails.
Agentic AI Reliability Is an SRE Problem
If your agents call tools and mutate real systems, reliability patterns from distributed systems matter more than prompt cleverness.
TCP Retransmits: The Quiet Fire Alarm in Your Metrics
If p99 is drifting and dashboards look normal, retransmits are often the first honest signal.
Readback, Repeatback, and the 2AM Page
Reliability isnât just systems design; itâs communication design under stress.
Uptime Is a Sleep Problem
If your reliability plan ignores sleep, it is quietly training your team to fail at 2 a.m.
Immutable Infrastructure, Without the Religion
Immutable systems reduce deployment drift and blast radius, but they work best when paired with pragmatic escape hatches.
Your Backup Is a Rumor Until You Restore It
GitLabâs 2017 outage is a reminder that backup success logs are not the same thing as recovery readiness.
When the Falcon Blinked: A Tragedy in Blue
One content update, 8.5 million broken Windows machines, and an entire industry relearning humility.
Immutable Is Cheaper Than Heroics
The Knight Capital outage is still the clearest argument for immutable infrastructure.
The Two-Minute Pause That Prevents the Long Night
What SRE teams can learn from cockpits and operating rooms about small rituals that prevent big failures.
The Pager Has a Heartbeat
Uptime is a human system, and sleep is part of the architecture.
The Bug Waited 26 Days
Two famous outages, one quiet lesson: incidents often start long before the pager goes off.
Time: The Invisible Dependency That's Been Quietly Wrecking Your Stack
NTP is 40 years old, unsexy, and quietly holding your entire distributed system together. Here's what happens when it slips.
When the Safety Net Eats the Trapeze Artist
The Facebook outage of October 2021 wasn't about BGP. It was about what happens when your safety mechanisms assume partial failure â and you get total failure.
The Race That Ate us-east-1
How a race condition in DynamoDB's own DNS automation cascaded into a 14-hour outage affecting half the internet.