AI-Generated Exploit Discovered by Google
Researchers at Google Threat Intelligence Group (GTIG) have identified a zero-day exploit targeting a popular open-source web administration tool, which was likely generated using artificial intelligence (AI). The exploit could be used to bypass the two-factor authentication (2FA) protection in the web-based system administration tool.
The GTIG has high confidence that the adversary used an AI model to find and weaponize the vulnerability, based on the structure and content of the Python exploit code. The script contains an abundance of educational docstrings, including a hallucinated CVSS score, and uses a structured, textbook Pythonic format highly characteristic of large language models (LLMs) training data.
Characteristics of the Exploit
The nature of the flaw, a high-level semantic logic bug, is also indicative of AI systems' capabilities, as they excel at identifying such issues, rather than memory corruption or input sanitization problems typically uncovered through fuzzing or static analysis.
The large language model (LLM) used for the malicious task remains unclear, but Google rules out the possibility that Gemini was involved in the process. The company notified the software developer about the significant threat and took timely action to disrupt the attack.
Threat Actors' Use of AI
Google notes that this is not an isolated incident, as Chinese and North Korean hackers, such as APT27, APT45, UNC2814, UNC5673, and UNC6201, have been using AI models for vulnerability discovery and exploit development. Russia-linked actors have also been observed using AI-generated decoy code to obfuscate malware such as CANFAIL and LONGSTREAM.
In addition, Google highlights a Russian operation codenamed “Overload,” where social engineering threat actors used AI voice cloning to impersonate real journalists in fake videos promoting the anti-Ukraine narrative.
PromptSpy Backdoor and Gemini APIs
The PromptSpy backdoor for Android, documented by ESET earlier this year, is also highlighted in Google’s report for its integration with Gemini APIs for autonomous device interaction. However, Google found an autonomous agent module named "GeminiAutomationAgent" that uses a hardcoded prompt to enable the malware to interact with the device in an automated way.
The goal of the prompt is to assign a benign persona so it can bypass the LLM's safety features, allowing the malware to calculate the geometry of the user interface bounds and interact with the device in multiple ways. The malware also makes use of AI-based capabilities to replay authentication on the device, be it in the form of a lock pattern or a PIN.
Industrialization of Access to Premium AI Models
Google is warning that threat actors are now industrializing access to premium AI models using automated account creation, proxy relays, and account-pooling infrastructure. This development raises concerns about the potential for more sophisticated and widespread attacks in the future.
As the use of AI in cyber attacks continues to evolve, it is essential for organizations to stay vigilant and adapt their defenses to counter these emerging threats. The discovery of this AI-generated zero-day exploit serves as a reminder of the importance of proactive security measures and the need for ongoing research and development in the field of cybersecurity.
- Related Articles:
- Critical cPanel and WHM bug exploited as a zero-day, PoC now available
- Disgruntled researcher leaks “BlueHammer” Windows zero-day exploit
- 13-year-old bug in ActiveMQ lets hackers remotely execute commands
- Max severity Flowise RCE vulnerability now exploited in attacks
- CISA: New Langflow flaw actively exploited to hijack AI workflows
Source: BleepingComputer