Agentic AI Reliability Is an SRE Problem
If your agents call tools and mutate real systems, reliability patterns from distributed systems matter more than prompt cleverness.
#reliability
24 transmissions tagged #reliability
TCP Retransmits: The Quiet Fire Alarm in Your Metrics
If p99 is drifting and dashboards look normal, retransmits are often the first honest signal.
Memory Tiers Stop Failure Cascades in Multi-Agent Systems
Most agent failures are not single bad calls. They are memory propagation bugs. A tiered memory architecture contains damage, improves evals, and makes recovery tractable.
Recovery Loops Beat First-Pass Accuracy in Agentic AI
Production agents are judged by how they recover from inevitable mistakes. Design loops for diagnosis, bounded retries, and safe handoff instead of chasing one-shot perfection.
Readback, Repeatback, and the 2AM Page
Reliability isn’t just systems design; it’s communication design under stress.
Uptime Is a Sleep Problem
If your reliability plan ignores sleep, it is quietly training your team to fail at 2 a.m.
Agent Memory Is the Control Plane
If your agents forget state, they will eventually fail safe tasks unsafely. Treat memory and retrieval as first-class control systems.
Why Multi-Agent Systems Fail at Handoffs (and How to Fix Them)
Most agent failures are handoff failures. Contract-driven tools, scoped memory, and trace-based evals make multi-agent systems actually reliable.
Immutable Infrastructure, Without the Religion
Immutable systems reduce deployment drift and blast radius, but they work best when paired with pragmatic escape hatches.
Your Backup Is a Rumor Until You Restore It
GitLab’s 2017 outage is a reminder that backup success logs are not the same thing as recovery readiness.
Agentic AI Needs Contract Tests, Not Just Better Prompts
A practical blueprint for making tool-using agents reliable with schema contracts, simulation harnesses, and replayable incident response.
Agentic AI Needs Durable Execution, Not Just Smarter Prompts
Why idempotency, checkpointing, and replay matter more than prompt tweaks once agents start touching real systems.
The Two-Minute Pause That Prevents the Long Night
What SRE teams can learn from cockpits and operating rooms about small rituals that prevent big failures.
Multi-Agent Systems Fail at the Seams: Build Control Loops, Not Chat Loops
Most multi-agent failures come from handoff seams, not model quality. Here is a practical control-loop architecture for reliability under real workloads.
The Pager Has a Heartbeat
Uptime is a human system, and sleep is part of the architecture.
Agentic AI Evals That Catch Real Failures
A practical evaluation stack for tool-using agents: replay tests, adversarial suites, and decision-quality metrics that prevent production regressions.
The Bug Waited 26 Days
Two famous outages, one quiet lesson: incidents often start long before the pager goes off.
Multi-Agent AI Needs Distributed Systems Rules, Not Better Vibes
If your agent swarm coordinates through free-form chat alone, you have a distributed system with no transaction model. Here is the production-safe architecture.
Time: The Invisible Dependency That's Been Quietly Wrecking Your Stack
NTP is 40 years old, unsexy, and quietly holding your entire distributed system together. Here's what happens when it slips.
Retry-Safe Agentic Systems: How to Keep Tool-Using Agents from Double-Spending Reality
Most agent failures are not model failures. They are orchestration failures. Build retry-safe loops with idempotency, durable state, and failure-oriented evals.
One Trillion Databases, Zero Pagerduty Alerts
SQLite runs on more devices than any other database engine in history. You've never been paged about it.
Agentic AI Control Loops That Survive Production
A practical architecture for tool-using agents: planner/executor loops, bounded memory, measurable evals, and failure containment.
When the Safety Net Eats the Trapeze Artist
The Facebook outage of October 2021 wasn't about BGP. It was about what happens when your safety mechanisms assume partial failure — and you get total failure.
The Race That Ate us-east-1
How a race condition in DynamoDB's own DNS automation cascaded into a 14-hour outage affecting half the internet.