Should PMS code with agents?

TLDR Product's Kasper Jung is asking whether product management systems should lean on AI agents to do the coding work. The piece walks through the current state of LLM evaluation and benchmarking — how we're measuring whether these models can actually deliver, not just churn out code that looks right.

The question isn't theoretical. Product management has always been about choosing what to build and how to know it's worth building. Now the tools are adding agents that can write code, test it, and iterate. The real test is whether those agents can be trusted with the kind of judgment that separates shipping something good from shipping something that works.

Why this matters for us:
When the tools that run the companies we work for start coding with agents, the people who understand the work — not just the people who understand the tools — get to decide what actually ships.