The USENIX 2025 Study
| Metric | Value |
|---|
| Models tested | 16 |
| Code samples analyzed | 576,000 |
| Overall hallucination rate | 19.7% |
| Open-source model rate | 21.7% |
| Commercial model rate | 5.2% |
| Unique hallucinated names | 205,000+ |
Open-Source vs Commercial Models
Open-source models hallucinate 4.2x more than commercial models (21.7% vs 5.2%). This is because commercial models have better training data curation and may have registry verification built into their training pipeline.
The Attack Chain
1. Developer asks AI to write code
↓
2. AI recommends a package (e.g., "react-codeshift")
↓
3. Package doesn't exist (19.7% chance)
↓
4. Attacker registers "react-codeshift" on npm
↓
5. Attacker fills it with malicious code
↓
6. Developer runs: npm install react-codeshift
↓
7. Malicious code executes in developer's environment
↓
8. Attacker gains access to source code, credentials, CI/CD
Real Incidents
react-codeshift: An AI-hallucinated package name that was forked into 237 repositories, translated to Japanese, and re-executed by other AI agents. The hallucinated name propagated through the ecosystem.
unused-imports: A malicious npm package with 233 weekly downloads. It exploited a name similar to a legitimate package, targeting developers who mistyped or accepted AI recommendations without verification.
Why slopsquatting is uniquely dangerous for AI code
Slopsquatting is uniquely dangerous for AI-generated code because: (1) AI hallucinations are consistent -- the same model will recommend the same hallucinated package name across different conversations, creating a predictable target for attackers. (2) AI code is often executed without human review -- developers trust AI recommendations and run npm install or pip install without checking if the package exists. (3) AI agents can propagate hallucinated names -- when one AI agent generates code with a hallucinated import, another AI agent may copy and re-execute it, spreading the hallucinated name across repositories. (4) 205,000+ unique hallucinated names give attackers a massive attack surface -- they can register hundreds of names and wait for any one to be installed.
The USENIX 2025 study found that 19.7% of AI-recommended packages don't exist, with 205,000+ unique hallucinated names. How does the slopsquatting attack work?
The slopsquatting attack works in 4 steps: (1) AI hallucinates a package name when generating code. The USENIX 2025 study analyzed 576,000 code samples from 16 models and found that 19.7% of AI-recommended packages don't exist -- the AI invented the package name. These hallucinations are consistent: the same model will recommend the same hallucinated name across different conversations, creating a predictable target for attackers. (2) Attacker registers the hallucinated name on a package registry (PyPI, npm, etc.) with malicious code. This is a passive attack -- the attacker doesn't need to actively target anyone. They register the name and wait for a developer to install it. With 205,000+ unique hallucinated names, attackers have a massive attack surface. (3) Developer runs pip install or npm install with the hallucinated name. This happens because developers trust AI recommendations and often install packages without verifying they exist. In AI-assisted coding workflows, the AI may even suggest the install command, making it easy for the developer to execute without thinking. (4) Malicious code executes in the developer's environment, giving the attacker access to source code, credentials, CI/CD tokens, and other sensitive data. The malicious package can exfiltrate data, install backdoors, or modify the build process. Open-source models hallucinate 21.7% vs commercial 5.2% (4.2x more), making open-source AI code more vulnerable to slopsquatting. The react-codeshift incident showed how hallucinated names propagate: the package was forked into 237 repositories, translated to Japanese, and re-executed by other AI agents."