The agent got 48 of 54 right. Three of the wrong ones published themselves.
An analytics agent answered 48 of 54 questions correctly on a local model. What mattered was 3 of 37 auto-published answers wrong, and why no check saw them.
11 min read
Topic
AI agents in production
Agents whose outputs land in typed, audited tables rather than chat transcripts. LangGraph, CrewAI, Google-ADK, MCP, structured contracts and fallbacks.
I write about the boundary between a useful AI demo and a system another workflow can trust. The central rule is simple: an agent’s output should become a typed, validated, traceable record before it changes a product or a business process. The hard work is rarely a clever prompt. It is tool design, bounded permissions, evaluation, provenance, fallback behavior, and deciding what the system must do when the model is confidently wrong.
7 posts
All writingThe eval score fell 4.8 points. Twelve answers decided whether it meant anything.
A prompt change cost 4.8 points on an LLM judge, and 12 of 83 paired cases carried all the evidence. Why a release gate needs a paired test, not a delta.
13 min read
Agents read your definitions. Somebody has to own them.
Agents fail on stale or contradictory knowledge before they fail on reasoning. One concept per file, an owner, a date, a Disputed section, and generate the rest.
13 min read
The user's laptop is not a server. Make it a queue consumer.
A cloud agent that drives a browser on a user's Mac kept reporting it offline. The fix was not sticky routing: model the laptop as a queue consumer.
13 min read
Ten gates, and the failure behind each one
Every gate in this repository exists because something got through. Three of them because a suite reported success against a broken artifact.
10 min read
Agents write the diff. The repo has to review it.
How to run Cursor, Claude Code, and Codex in an existing repo: AGENTS.md, CODEOWNERS, CI gates, and the AI-generated PR pitfalls that blow up review.
16 min read
The data platform is part of the product
A practical operating model for reliable data and AI products: consumer-facing promises, typed decisions, idempotency, provenance, and repair.
11 min read