Verifying AI models with math, not just tests

Georg Wiese is writing about formal verification for AI — proving things about models with math instead of running them through tests. Tests tell you what happened; proofs tell you what must be true. That's the difference between checking a lock worked and knowing the lock can't fail.

The key idea is that you can write a specification — a mathematical statement about how the model should behave — and then use tools like Coq or Lean to prove the model satisfies it. If the proof goes through, you're done. No more wondering whether the test suite covered the edge cases.

This isn't new in theory, but it's new in practice for AI. Most verification has been for compilers and protocols; now people are trying it on neural nets. The work is still early — the proofs are expensive, and the models are bigger than the specifications — but the direction is clear. If you want to trust a model, formal verification is the way.

Why this matters for us: when the models break in production, the Brown families and small shops who depend on them need to know whether a failure is a fluke or a structural problem — formal proofs tell us which.