Anthropic's new research tool is trying to fix AI safety

Anthropic just published a paper on a research assistant called Claude 4 Sonnet that can read academic papers, write code, and run experiments all on its own. The goal is to scale up the kind of work researchers do when they're looking for alignment failures in language models—finding edge cases where a model does something it shouldn't, like lying about its capabilities or refusing to follow instructions when it should.

The system is set up so the model proposes a test, writes the code for it, runs it, then reads the results and decides what to try next. It does this iteratively, looping through hundreds of attempts, trying to surface the kinds of failures that make models unsafe or unpredictable. Anthropic says this approach catches failures that a single round of testing would miss.

The bigger picture here is that as models get more capable, the kinds of problems they can have are getting harder to find. You can't just sit there and prompt them anymore. You need systems that can reason about what to test and follow up on results. This is one of several labs trying to automate that kind of work. OpenAI has been doing something similar with their own research agents. The question isn't whether models can do this work—it's who's doing it, and for what.

Why this matters for us: the people building these systems are deciding what counts as a real failure, and we're not in the room when they make those calls.