As AI-generated code becomes increasingly prevalent, concerns about the expanding attack surface for malicious hackers have grown. While some argue that large language models can enhance cybersecurity capabilities by finding and fixing vulnerabilities quickly, new research suggests that AI-generated patches may not be reliable.
Research Findings
Researchers at 1Password tested the patching capabilities of two commercial models, OpenAI's ChatGPT 5.5 and Anthropic's Claude Opus 4.8, on six high-impact, high-complexity CVEs, including the 'Copy Fail' vulnerability. The results showed that the models had a low rate of successful patch generation, with a success rate of less than 47%.
According to the research, the models often addressed only a subset of vulnerable code paths, added fragile guard code that satisfied tests while failing to address the vulnerability's root cause, and sometimes introduced subtle changes in the application's behavior while patching the immediate vulnerability.
Veracode Report
A report from Veracode found that the average security 'pass rate' for AI-generated code is around 56%, with newer models like GPT 5.5 pushing closer to 70%. However, more than half of the models tested had a pass rate between 50-53%, and in 44% of the tests, the models introduced a detectable OWASP Top 10 vulnerability into the codebase.
Tim Jarret, vice president of product at Veracode, noted that AI tools are still subject to limitations that can make them unreliable for cybersecurity patching without human oversight. While some vulnerabilities can be easily patched through automation, others require a human touch or additional context to fully close off.
Limitations of AI-Generated Patches
Jarret emphasized that it is premature to treat AI-generated patches as anything other than another code change that needs to be reviewed and accepted by the team. He acknowledged that automated code review will be necessary, but it should not be performed by the same automation tool that produced the code.
The ultimate goal, Jarret said, is to 'trust but verify.' In most cases, a human check may simply involve verifying that the patch looks good, but in cases where there are still issues, more attention is needed.
As AI-generated code continues to grow, it is essential to address the limitations of AI-generated patches and ensure that they are reliable and effective in fixing vulnerabilities. This requires a combination of human oversight and automated code review to verify the patches and prevent the introduction of new bugs or vulnerabilities.
Future of AI-Generated Patches
While the research highlights the limitations of current AI-generated patches, it also notes that newer models, such as Anthropic's Mythos or OpenAI's GPT-5.6-Sol, may have significantly higher cybersecurity capabilities. These models can identify and fix vulnerable code, and companies are distributing them to key industries through initiatives like Project Glasswing and Daybreak.
As the use of AI-generated code continues to expand, it is crucial to develop more reliable and effective patching capabilities. This may involve improving the accuracy of AI models, enhancing human oversight, or developing new automated code review tools. Ultimately, the goal is to ensure that AI-generated patches are trustworthy and effective in fixing vulnerabilities, and to prevent the introduction of new bugs or vulnerabilities.
Source: CyberScoop