OpenAI Detects Coordinated AI Model Distillation Campaign
OpenAI announced on Wednesday that it had detected and disrupted a coordinated campaign aimed at distilling and extracting reasoning capabilities from its artificial intelligence models. The company stated that low-level activity was first observed on July 1, which gradually increased over the following weeks. By July 24 and 25, OpenAI recorded 16,000 prompts from 4,000 users exhibiting a consistent 'relevant extraction pattern.' The number of suspicious users rose to 15,000 by July 28, at which point OpenAI said it had 'fully disrupted' the operation.
Attackers Used Novel Method to Bypass Encryption
The attackers employed a technique OpenAI described as 'novel' to circumvent protections on reasoning data. Rather than breaking encryption or gaining direct access to stored conversations, the operators copied encrypted reasoning data from one conversation and then prompted the model in a separate conversation to decrypt and transcribe that content into plain text. This method allowed them to reproduce protected reasoning in a visible form without compromising underlying security systems.
The operators did not break our encryption, compromise a database, or gain direct access to stored user conversations. Instead, they manipulated model interactions so that protected reasoning could be reproduced in forms visible to the requester in a coordinated, scaled manner that violated our terms of service.
OpenAI shared these details in an unsigned blog post, emphasizing that no cryptographic breach or database compromise occurred.
Moonshot AI Identified as Core Actor in Campaign
OpenAI indicated that while it remains unclear whether all observed activity is connected, individuals working on behalf of Moonshot AI—a China-based rival AI company—were behind a 'core cluster' of the suspicious behavior. Moonshot AI has previously faced accusations of distilling U.S.-developed AI models to improve its own offerings. The company’s Kimi model is among several open-source Chinese AI models that provide competitive performance at low or no cost.
American AI firms and U.S. government officials have accused Chinese companies like Moonshot AI of conducting 'systematic' distillation attacks on advanced American models. Cybersecurity experts from Google and other firms have noted that such actors often acquire thousands of individual accounts through black or gray markets for services like Claude and ChatGPT, then flood them with millions of prompts to extract model capabilities and training data.
OpenAI Shares Details, Notes Broader Implications
OpenAI told CyberScoop it is not releasing additional technical information about the incident for security reasons. CyberScoop has reached out to Moonshot AI for comment, but no response was included in the original report. OpenAI stated that the same vulnerability exists in other AI models and confirmed it has shared information about the incident with groups such as the Frontier Model Forum.
Mitigation and Defensive Measures Implemented
Beyond banning the offending accounts, OpenAI said it strengthened its signup and infrastructure controls, expanded network monitoring, and fixed a specific bug that had allowed users to transfer encrypted data from one conversation and decrypt it in another. These changes were made to prevent similar abuse of model interactions in the future.
Source: CyberScoop