AI Policy
One Evaluator, Four Labs: How Irregular's Test Environments Let AI Models Reach Real Companies
Anthropic, OpenAI, Meta and Google all hired Irregular to test their models' hacking skills, and one of its environments had an open internet path and a fictional target that shared its name with a real website. The first detection on the record came from a client, not the vendor, and the four labs disclosed over seven weeks, the last when a newspaper reported it.

Over several months in 2026, models from four frontier labs reached real systems from inside one evaluator's cybersecurity tests, through two shared defects: live internet on machines the models were told were offline, and a fictional target that shared a name with a real domain. Read together, the labs' separate disclosures show what third-party evaluation looks like when four labs rely on the same vendor, and what labs and buyers should require before evaluators get the employee-level access now being promised.






