Open-source AI code scanners are getting real

Semgrep published a head-to-head comparison of open-source AI code-security harnesses. The post maps how these tools flag vulnerabilities in codebases that AI generates or modifies. It’s not a hype piece. It’s a side-by-side run.

The comparison is organized by what the tools actually do in practice: scanning, alerting, false-positive rates, and how they handle the messy middle ground where AI writes code that looks right but isn’t. The author walks through test cases — real-ish repos, real-ish prompts — and shows where each harness trips up. No vendor logos. No sponsorships. Just the output.

The post also notes the broader context: AI code generation is moving fast. OpenAI’s recent hacking incident, Starlink’s role in enabling a $114 billion crime network, and the general arms race around AI security all point to one thing — the tools we use to check the code aren’t keeping pace with the tools that write it.

The comparison doesn’t pick a winner. It picks the right tool for the job. Some harnesses are better at catching injection flaws. Others are better at catching logic errors. Some are faster. Some are more thorough. The author lays out the tradeoffs so you can stop guessing.

There’s no Silicon Valley jargon here. No “leverage.” No “ecosystem.” Just a developer looking at a repo, running a scanner, and seeing what shows up in the console. That’s the point. If your code gets flagged, you fix it. If it doesn’t, you’re still at risk.

The post is technical but readable. You don’t need a PhD in cybersecurity to follow it. You just need to care that the code you ship doesn’t leave your users exposed.

Why this matters for us:
When AI writes code for our apps, our clients, our side businesses, we need tools that actually catch the mistakes before they hit production — and this comparison tells us which open-source options are worth running in our own pipelines.