Insights
Notes from the work, not the hype.
Practical thinking from the engineers and operators replacing manual work with systems that actually ship in production.
Most Multi-Agent Systems Should Be One Agent and a State Machine
Most multi-agent architectures we're asked to build should be one scoped agent plus deterministic orchestration. When handoffs earn their keep, and a test to decide.
Read more
Naive Retries Are Quietly Wrecking Your Production Agent
Naive retry logic re-runs tool calls that already mutated state, hides prompt failures, and burns budget. A decision framework for when to retry, reprompt, escalate, or fail loud.
Read more
Your RAG Leaks: Permissions Are the Feature You Skipped
Naive vector search surfaces content the asking user can't see. The ACL patterns we use in enterprise RAG — pre-filtering, post-checks, per-user indexes — and their tradeoffs.
Read more
Your RAG Index Needs Schema Discipline, or It Lies
A RAG knowledge base needs the same versioning, migration, and rollback discipline as a database schema. Here's what breaks when you skip it.
Read more
Schema-Valid and Still Wrong: The False Confidence of Structured Output
Forcing LLMs into JSON schemas catches type errors, not lies. The validation layer we wire in beyond the schema — with a transferable checklist.
Read more
When a Reranker Earns Its Latency (and When It Doesn't)
A candid guide to when a cross-encoder reranker actually fixes retrieval quality versus when chunking or hybrid search is the cheaper fix — with a decision heuristic and before/after numbers.
Read more
Budget Latency Across the Whole Agent Path, Not Just the Average
Why p99 matters more than p50 for agent trust, and how to budget latency across retrieval, tool calls, inference, and streaming to fix the long tail.
Read more
Grading Actions, Not Answers: Evals for Agents That Do Things
Why evaluating an agent that sends emails and updates records is harder than grading text — and how to build eval harnesses that assert on side effects.
Read more
Stop Fine-Tuning to Fix a Retrieval Problem
Most teams reach for fine-tuning to fix problems that are actually retrieval or prompting. A candid decision checklist, with the two times fine-tuning genuinely won.
Read more
Your RAG Cited a Real Source and Still Lied to You
Fluent RAG answers cite a real chunk and still get it wrong by overreaching past the passage. The faithfulness checks we wire in to catch it.
Read more
Your Agent's Real Failure Rate Isn't in the Ticket Queue
Complaint rate is a terrible proxy for agent quality. Here's the instrumentation we use to estimate the silent failure rate most teams never measure.
Read more
Your RAG Cites the Wrong Policy Because Your Docs Disagree
Retrieval works, the corpus doesn't. How outdated, duplicated, and contradictory docs make agents confidently cite the wrong policy — and how to surface the conflict.
Read more
Define the Acceptance Test Before the Build, Not After
Why we write a concrete, measurable acceptance test with clients before an 8-12 week AI build starts — and how it kills projects that shouldn't exist.
Read more
The Agent That Passed Every Eval and Rotted Anyway
Agents ship passing evals, then quietly degrade over weeks as prompts get tweaked and vendor models update. How to build CI-style regression gates that catch it.
Read more
Design the Refusal Boundary Before Your Agent Deletes a Record
How to define the actions a production AI agent must never take alone — and enforce it with tool permissions and validation gates, not prompt pleading.
Read more
The Trace You Wish You Had After an Agent Fails in Prod
The span-level tracing and attributes we instrument on every agent before go-live — so a failed answer takes minutes to localise, not a day.
Read more
Human-in-the-Loop That Doesn't Train Reviewers to Rubber-Stamp
Approve-every-action gating trains reviewers to click yes. How to gate on uncertainty, blast radius, and reversibility instead — with a decision table.
Read more
The Data Access Layer Is Where Agent Projects Actually Stall
Enterprise agent projects don't stall on model choice or prompts. They stall on stale ETL, row-level permissions, and no clean API. A CTO's de-risking checklist.
Read more
Document Chunking for RAG: What We Dropped and What We Ship
The chunking strategies we tried and abandoned across messy enterprise corpora — and the structure-aware, metadata-rich approach we default to now.
Read more
Prompts Are Artifacts, Not Dashboard Config: How We Version Them
How we version, eval-gate, canary, and roll back prompts in production AI systems — and why editable dashboard prompts silently degrade quality across a fleet.
Read more
Your RAG Isn't Hallucinating — It Never Saw the Answer
Teams grade answer quality while ignoring whether the right passage was even retrieved. How to build a retrieval-only eval and why it comes first.
Read more
Five RAG Failure Modes We Keep Finding in Audits
A candid field guide to the five RAG failure modes we keep diagnosing in client audits — symptom, real root cause, and the cheapest fix for each.
Read more
How We Stress-Test an AI Agent Before It Ships
The harness we use to break an agent before production does: adversarial inputs, failure injection, tail latency, cost per resolved task, and injection probes.
Read more
How We Cut an Agent's Token Bill 60% Without Losing Eval Points
An anonymised case study: where an agent's tokens actually went, the changes that cut spend ~60%, and the measurement discipline that proved quality held.
Read more
How We Shipped a Support Agent That Deflected 40% of Tickets
An anonymised case study of a customer-support agent that deflected ~40% of inbound tickets without escalation-rage — the failures, the fixes, and the deflection ceiling.
Read more
Catching Agent Failure in Production Before Your Users Do
How to detect AI agent failure in production before users notice — drift detection, online monitoring, confidence-based fallbacks, and the telemetry that makes it work.
Read more
Workflow or Agent? A Decision Framework for Enterprise AI
Most enterprise tasks handed to us as 'agent' projects ship better as constrained workflows. Concrete criteria for deciding when to give a model agency — and when to take it away.
Read more
Why Your Agent Makes 40 Tool Calls to Do a 6-Call Job
Production agents balloon their tool-call counts with redundant retrievals and second-guessing loops. How to instrument, budget, and detect thrashing.
Read more
Why LLM Confidence Scores Lie — and Break Your Human Gate
LLM self-reported confidence and logprobs are badly calibrated. Here's how that breaks human-in-the-loop gating in production agents — and the cheap fixes that work.
Read more
Golden Eval Datasets Rot: How to Keep Yours Honest
Golden eval sets silently rot within months. How we source real edge cases, version evals alongside prompts, and refresh without invalidating history.
Read more
The Agent That 10x'd Traffic and Blew Its Budget
An anonymised case study of an AI agent that worked in pilot but blew the budget at scale. Where the cost went, the fixes, and the before/after numbers.
Read more
The Model Writes the Bug and the Confidence
AI-generated code ships exploitable flaws with total confidence. Why security review is the one step you can't delegate to the tool that caused the problem.
Read more
Treat AI Agents Like Untrusted Insiders, Not Magic
DeepMind's AI Control Roadmap signals the shift from hoping models stay aligned to securing agents like insider threats. Here's the executive mental model.
Read more
The Eval Harness Is the Deliverable, Not the Agent
For agentic systems, the eval harness is what lets you ship with confidence and iterate safely. Here's what goes into a real one — and why teams pay for skipping it.
Read more
Why Your RAG Demo Works and Production Doesn't
The specific failure modes that kill RAG systems after the demo — retrieval drift, chunk boundaries, stale indexes — and how to catch them with evals first.
Read more
Build vs Buy: AI Agents for Enterprise
A decision framework for CTOs and Heads of AI in 2026. Four paths — Build, Buy, Partner, Wait — with honest costs, timelines, and failure modes.
Read more
Why Most Enterprise AI Pilots Die in Month 4
The month-four death has remarkably consistent causes. None of them are technical. Four organisational failure modes that kill AI pilots.
Read more
How Anglo American Cut Fleet Downtime 40% with Predictive Maintenance
Case study: 100+ vehicles, 2 years, 40% downtime reduction. What we built, why it worked, and the patterns worth stealing.
Read more
The Anatomy of a 6-Week Enterprise AI Project
There is a specific shape that enterprise AI projects take when they actually ship. Here's the week-by-week breakdown.
Read more
Buy vs Build vs Fractional: A Decision Framework for Enterprise AI
The question is almost never framed correctly. A framework for deciding how to stand up AI capability — honest about where each option breaks.
Read more
AI Automation for Enterprise: What Actually Works in 2026
Most enterprise AI projects fail. Here's what separates the ones that ship from the ones that stall — based on what we've seen across 6 industries.
Read more
Why More Companies Are Hiring a Fractional Head of AI
You need senior AI leadership but aren't ready for a full-time hire. Here's how the fractional model works and when it makes sense.
Read more