Verified as of July 7, 2026. This review is scheduled for quarterly updates to reflect rapid changes in AI ethics regulations and corporate safety practices.
- The Mechanics of Operation Cannes
- How Competitors Responded to the Testing
- Meta’s Official Defense
- The Broader Implications for AI Safety Benchmarking
- Separating Safety Probing from Broader Tech Conflicts
- FAQ
- Sources
A covert testing program conducted by Meta has triggered significant ethical questions across the technology industry. According to an investigative report published by Wired, Meta hired external contractors to create fake accounts listing ages under 18. These disposable accounts were then used to send crisis-style prompts (including suicide, self-harm, sex, drugs, and eating disorders) to rival artificial intelligence chatbots without the consent or knowledge of the targeted companies.
The targeted platforms included OpenAI’s ChatGPT, Google’s Gemini, and Character.AI. While tech companies regularly run internal safety stress tests, secretly posing as minors to probe competitor safety guards represents a highly adversarial approach to AI benchmarking.
The Mechanics of Operation Cannes
The covert project was codenamed “Cannes” internally and run through Covalen, an Irish outsourcing and contracting firm. Under Operation Cannes, contractors created burner accounts on rival platforms, setting the user profiles to mimic children or teenagers under 18.
The scale of the project was extensive. A single round of testing in August 2025 involved more than 45,000 prompts across 3,748 distinct prompt variations. Of these prompts, at least 239 explicitly referenced sex or romance. Contractors were instructed to log the chatbot responses in spreadsheets to evaluate how well rival guardrails held up under pressure.
In addition to text-based prompts, contractors uploaded visual content to evaluate image processing. This included images of pills, knives, nooses, and a medical illustration of a gynecological procedure. The goal was to check if competitor models could be coaxed into bypassing their own safety filters when presented with visual crisis cues. The project remained active as of April 2026.
How Competitors Responded to the Testing
The targeted companies (OpenAI, Google, and Character.AI) were not informed about Project Cannes. When the details were exposed, their responses varied. Character.AI specifically stated that the conduct violated its Terms of Service. OpenAI stated it was looking into the matter and declined further comment. Google stated it had not approved the testing and did not know its purpose, adding that its own audits showed Gemini responded in accordance with safety policies.
The safety of under-18 users has been a sensitive topic for these platforms. For instance, the ongoing Character.ai teen safety debate highlighted the intense public pressure on AI companies to shield minors from harmful or parasocial interactions. Meta’s adversarial tests focused directly on these sensitive areas, probing how easily rival platforms could be forced to output harmful self-harm or adult advice.
Meta’s Official Defense
Meta did not deny the existence of Project Cannes. In a statement provided to the media, a Meta spokesperson defended the project:
“Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice, and any suggestion otherwise completely misunderstands how technology companies work to refine and improve their systems.”
The spokesperson also stated that Meta does not use the data gathered from this competitor benchmarking to train its own AI models. Instead, Meta frames the testing as a way to understand the baseline safety levels across the wider industry.
The Broader Implications for AI Safety Benchmarking
Project Cannes exposes a significant gap in how AI safety is evaluated, emerging at a critical regulatory moment. In September 2025, the Federal Trade Commission (FTC) launched a formal inquiry into artificial intelligence and child safety, specifically targeting Meta, OpenAI, and Google. The revelation that Meta was secretly probing its competitors using fake minor accounts during an active federal safety inquiry raises the regulatory stakes.
Currently, there is no standardized, independent body that benchmarks how AI chatbots handle sensitive inputs. Instead, companies rely on adversarial testing (often called red-teaming) to find vulnerabilities. When red-teaming is conducted internally on a company’s own model, it is considered best practice. However, when a company secretly targets competitors using fake minor profiles, the line between safety research and corporate intelligence becomes thin. Critics point out that by using under-18 burner accounts, Meta bypassed standard developer agreements, utilizing crisis prompts that could trigger real-world alerts on competitor systems.
Separating Safety Probing from Broader Tech Conflicts
It is important to keep the Operation Cannes controversy distinct from other recent technology disputes. This safety probing project is entirely separate from the Fable 5 export-control investigations, which focus on national security and hardware restrictions. It is also unrelated to the Anthropic-Alibaba intellectual property disputes, which involve code theft and copyright claims.
Project Cannes is a case of adversarial benchmarking and corporate safety probing, raising questions about tech ethics, user terms, and how companies evaluate safety in a highly competitive market.
FAQ
What was the codename of Meta’s secret testing project?
The project was codenamed “Cannes” internally and was operated through the external contracting firm Covalen.
Which rival AI platforms were targeted by Meta’s contractors?
The contractors targeted OpenAI’s ChatGPT, Google’s Gemini, and Character.AI.
What kind of prompts did the contractors send?
Contractors sent crisis-style prompts about suicide, self-harm, eating disorders, drugs, and sexual relationships, often accompanied by images of pills, knives, nooses, and a medical illustration of a gynecological procedure.
Did Meta use the competitor data to train its own models?
No. Meta stated that the data collected from Project Cannes was used strictly for safety benchmarking and was not used to train Meta’s own AI models.
Is adversarial safety testing illegal?
While safety benchmarking is common, using fake accounts to probe a competitor’s system without consent generally violates that platform’s terms of service. However, whether this constitutes a legal violation or is classified as standard competitive research remains a gray area in technology law.
Sources
- Original Wired Investigation: Wired
- Meta Corporate AI Safety Principles: Meta AI
- Google Gemini Safety Guidelines: Google Safety
- OpenAI Usage Policies: OpenAI
About the Author
Ether Exter is an AI enthusiast with 5 years of experience testing and experimenting with AI models, breaking down what actually works. Follow on X: @EtherExperiment.
