How to test LLMs before shipping — the practical checklist

A new paper from OpenAI and researchers at the University of California, Berkeley maps out how to evaluate language models before they hit production. The authors walk through the full pipeline: picking the right test set, choosing evaluation metrics, running benchmarks, and checking for failure modes like hallucination, bias, and prompt injection. They flag that accuracy isn't the only metric — latency, cost per token, and safety all matter for real systems.

What stands out is the part most teams skip: the adversarial test set. You don't just run the model on normal prompts; you run it on the prompts it's likely to get wrong — edge cases, multi-step reasoning, edge-case inputs. They show that models which look fine on standard benchmarks can still fail hard in production. The paper also covers how to detect when a model is confidently wrong, and how to set up human-in-the-loop review for the cases that need it.

This isn't theory. The authors have tested dozens of models using this framework and the differences are real — some models score well on standard benchmarks but collapse on adversarial tests, others are slower and more expensive for marginal gains. The takeaway is practical: if you're building with LLMs, you need a test set that looks like what your users will actually send, not just the standard leaderboards.

Why this matters for us: small shops and solo builders using OpenAI or Anthropic APIs are the ones most likely to skip these tests — and the ones who get burned when the model starts hallucinating in front of real customers.