Evals

Data & evals layer4 essays

How to know whether an AI feature is actually good, before and after launch.

Essays on Evals

September 20261 essay

26Sep 2026

Set a failure budget before you read the eval score

A single pass rate hides which failures your product is accepting. Turn an eval into a failure budget with explicit limits, owners and release decisions.

#evals#failure analysis#release criteria

August 20262 essays

22Aug 2026

Evals2 min read

The interface for agentic ambiguity

When AI agents operate in ambiguous territory, the interface needs to explicitly expose the trade-offs, not just the output.

#agents#ai ux#ambiguity

01Aug 2026

Evals3 min read

Why your LLM tests miss edge cases

Testing LLMs like software breaks when the model finds paths you didn't anticipate. A framework for finding the gaps in your eval set.

#evals#testing#llm

July 20261 essay

11Jul 2026

Evals5 min read

Your eval set is the real PRD

For AI features, the document that actually decides what ships isn't the spec. It's the list of inputs and the answers you'll accept. Write that first.

#evals#prd#llm

New essays every Tuesday and Saturday.

Follow along on LinkedIn, or get every essay from the subscribe page. No spam, no tracking.

Subscribe Follow on LinkedIn