Felony Bench
Felony Bench
A benchmark you really don't want models to be saturated with.
ModelEvaluator
Score
↖ Most illegalLeast illegal ↘
8
Anthropic
8
OpenAI
1
Meta
0
0
Moonshot
Scores indicate count of illegal activity. Higher is... you decide.
| Company | Felonies | Description | Date | Source | | --------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Anthropic | 1 | Exploited auth failures in an API to cancel other people's gym classes | 8/9/2026 | ABC Australia ↗ | | Meta | 1 | Compromise of an internal account at one company | 8/5/2026 | The Information ↗ | | Anthropic | 4 | Unauthorized use of GitHub credentials; Dependabot supply-chain attack; social engineering email campaign; public exposure of a malicious DNS server | 8/4/2026 | AISI ↗ | | OpenAI | 2 | Unauthorized use of GitHub credentials; public exposure of a malicious DNS server | 8/4/2026 | OpenAI ↗AISI ↗ | | OpenAI | 1 | Compromise of an internal account from a misconfigured CTF evaluation | 8/4/2026 | OpenAI ↗ | | OpenAI | 4 | Compromise of internal accounts at four companies as part of the Hugging Face incident | 7/31/2026 | OpenAI ↗Reuters ↗ | | Anthropic | 3 | Compromise of internal accounts at three companies | 7/30/2026 | Anthropic ↗ | | OpenAI | 1 | Compromise of Hugging Face during a model evaluation | 7/21/2026 | OpenAI ↗ |
Methodology
Felony Bench counts unique instances where AI agents affect third-party entities. Escaping a sandbox alone does not constitute a counted incident. It is for these reasons that Frontier Security's Kimi K3 incident and Alibaba's ROME incident are not counted.