From Mexico, I keep hearing two stories about AI and open-source security, and they sound like they can’t both be true. One says maintainers are drowning in junk bug reports written by chatbots. The other says AI models are now finding real, serious bugs faster than anyone can fix them. On October 8, Anthropic’s Frontier Red Team published a post that basically says: both happened, in that order. And it launched a service built around the second one: “Launching an opt-in vulnerability-finding service for open-source software”.
The service is called OSS Scanner. The pitch is short: “Projects that join will receive thorough, periodic security scans by our strongest models at no cost.” What makes it interesting isn’t the free part. It’s what Anthropic is willing to send you, and what it’s asking you to accept in return.
The models got faster than the reviewers
Anthropic opens with a benchmark. On CyberGym, an academic vulnerability-finding benchmark, “LLMs have gone from finding under 20% of vulnerabilities at the beginning of last year to finding over 85% this year.” As a result, the post says, open-source maintainers have gone from receiving “mostly slop” from LLMs to receiving high-quality bug reports.
Then comes the number that explains the whole launch. Over the last six months, Anthropic says it scanned some of the world’s most important software projects and found over 29,000 candidate vulnerabilities. Its people were able to manually review and triage about 6,000 of them. In the post’s own words, “we remain bottlenecked on our human capacity to validate these findings.”
That gap is the story for me. The bugs aren’t the scarce thing anymore. Careful human attention is. And some maintainers already made their choice: Anthropic says that, more and more often, maintainers who get its first reports simply ask for everything, unverified, with proposed patches. To date it has sent nearly 5,000 reports directly to maintainers who asked for the full pile.
A fast lane, with the label on the box
OSS Scanner turns that informal request into a program. Anthropic will keep sending human-verified reports through its usual coordinated disclosure process, especially to projects that don’t have the people to triage things themselves. OSS Scanner is the optional fast track next to that. And the post is blunt about the trade: “The outputs of this opt-in vulnerability scanner will be fully model-generated, without human review or triage.” That means faster and more frequent scans, but also reports that could be wrong.
Each report, Anthropic says, comes with a self-contained reproducer, an explanation of the bug (with a bisection to find when it was introduced, where possible), and a candidate patch when one is available. The reports come from its strongest models, including Claude Mythos, and the post says the service was inspired by Google’s OSS-Fuzz, which scans open-source code with fuzzers.
How good are unreviewed reports?
This is the question I’d ask first, and Anthropic gives a real answer. To test an early version, it had the penetration testers who review its disclosure findings check 97 critical and high-severity vulnerabilities from the scanner, across 48 projects. Of those, 85 (88%) met the bar for its disclosure process. Of the other 12, 11 were real but duplicated known issues or other findings, and just one was a false positive.
The maintainer quotes in the post line up with that. Todd Ouska of wolfSSL says “of the 74 reports we received, all but two were valid, and five became CVEs.” Daniel Stenberg says OSS Scanner helped the curl team find multiple issues, “including one of the worst curl vulnerabilities reported in the last few years.” Noah Misch of PostgreSQL says several reports came with fixes they could use nearly as-is.
To be fair, these are hand-picked quotes in a launch post, and Anthropic says so itself in a way: “Some have told us severity ratings can be inflated or the scanner misunderstood the project’s threat model.” Then: “We can’t guarantee the scanner will be perfect”. I trust a launch post more when it says that out loud.
How a project signs up, and who it’s for
The OSS Scanner FAQ has the practical details, and they’re very developer-shaped. Core maintainers enroll by opening a pull request to the anthropics/oss-scanner repo that adds a project.yaml file. The config needs the repo to clone, a primary contact email, and a Dockerfile that installs every dependency and builds the project “so that a fully offline agent can conduct its security audit.” Anthropic says it runs its scanning agents only after fully disabling internet access, inside hardened sandboxes.
My favorite part is the optional threat model file. Maintainers can describe what code should be tested, which inputs count as hostile, what’s out of scope, how they want severity rated, and what kind of patches they’d like. That’s the same context I’d give a new human contributor, and it’s a good habit even if you never enroll: if an AI tool doesn’t know your threat model, it’ll guess.
Eligibility follows criteria similar to OSS-Fuzz: projects with a “critical impact on infrastructure and user security”, decided case by case. The FAQ is also honest about who should not rush in: “This service is built for projects that are already able to keep up with verified high/critical vulnerability reports and are now looking to further secure their code.” Reports go out by email. A project can pause with disabled: true in its config, or leave by deleting its folder through another pull request. And because the findings are unvalidated, the FAQ says “We will not place any form of 90-day coordinated disclosure period on these unvalidated findings.”
What I take from it
For years, the worry with AI and open source was that it would make noise. This launch is about the opposite problem: the signal is good enough now that human review is the slow part, so the review step becomes a choice. I think that’s the right shape for it. Nobody gets the firehose by default. A maintainer has to ask for it, describe the project, and can turn it off.
It also changes what using AI in software engineering looks like for a lot of us. It’s not only the agent writing code in my editor. It’s an agent somewhere else reading my code, building it offline, and mailing me a patch. If you maintain something important, the work moves from finding bugs to judging and merging fixes fast. If you don’t, writing down your threat model is still the cheapest security upgrade on this list.
Featured image: Cologne Germany Hose-Coil-01, September 2014, photo by CEphoto, Uwe Aranas, Wikimedia Commons, CC BY-SA 3.0. Cropped, resized to 1400×900, and lightly adjusted from the original upload; this adapted image is shared under the same CC BY-SA 3.0 license.
