Introduction to AI Agent Insider Threats
Government agencies, cybersecurity companies, and threat researchers are studying how AI tools can be used by malicious actors to hack into organizations. However, as AI becomes more embedded in business infrastructure, there is a growing concern that breaches could be caused by insiders guiding these tools, either maliciously or due to lack of security controls.
Research by DTEX highlights the risks associated with AI agents, such as Anthropic's Claude Cowork, which is used in corporate environments. Claude Cowork includes tools that allow users to remotely control their agents, including a plugin for communicating with Salesforce AI agents that access and transfer data.
Testing AI Agent Capabilities
DTEX researchers tested two scenarios using Claude Cowork. The first scenario involved prompting Claude to summarize information from Salesforce and paste it into a draft Outlook email. The second scenario tasked the agent with archiving selected files and transferring them via the Cowork app.
In both cases, researchers used simple, single-turn prompts and spent between 10-30 minutes preparing to exfiltrate the data. The tests confirmed that the agents had access to sensitive systems, applications, and data, including the ability to download SharePoint corporate data, production documentation in OneDrive, access to Outlook email, Salesforce data, and any other files on the user's endpoint device.
Security Implications
Alex Desmond, director of insider threat intelligence and innovation at DTEX, notes that improvements in AI models and deeper integration of AI tools into IT network operations have reduced the time defenders have to react to a breach. "In cyberattacks, you talk about the kind of execution time of adversaries coming in and dropping ransomware, we're now seeing the kill chain drop to 30 and 10 minutes depending on what they're doing," Desmond said.
The speed and direct access to business networks or cloud services can create an insider threat nightmare for organizations. Western IT and cybersecurity businesses have been infiltrated by job applicants secretly working on behalf of the North Korean government, who use their salaries to evade international sanctions and fund Pyongyang's nuclear program.
Insider Threat Risks
Desmond warns that giving these individuals access to AI tools on top of their existing access can make it easier for them to steal sensitive data or assets. "You've got a nation-state actor getting into an environment legitimately," Desmond said. "Now if you gave them access to AI tools on top of that…you're like 'here's the keys to everything and here's this awesome tool that's just going to make your job – stealing our data – easier.'"
The tests by DTEX confirmed that the agents indeed had access to sensitive systems, applications, and data. For each of these applications, Claude Cowork has a dedicated plugin or API to share externally if prompted.
IT Governance and Visibility
The research by DTEX does not involve exploiting a software bug or configuration vulnerability, and it does not come with a CVE. Instead, it highlights an IT governance and visibility problem. Businesses are racing to integrate AI tools into their workflow and pushing employees to use the technology while failing to put in place the kind of security controls, access policies, and monitoring required to spot problems.
For instance, it may not be possible to determine how a data breach or leakage involving an AI agent actually occurred if an organization is not logging and auditing its prompts – or whether the incident was the result of an agent running amok or responding to potentially malicious instructions.
While network and cloud monitoring can identify when data is being accessed or downloaded from SharePoint, that may not be a strong enough signal to stand out for defenders. "If a user's normal workflow is to pull sensitive files down to work locally all the time, you don't have endpoint monitoring and you introduce an AI agent, it then just has access to all that data" along with the ability to exfiltrate it, Desmond said.
- AI agents can be used to exfiltrate sensitive data if not properly monitored and controlled.
- Businesses are racing to integrate AI tools into their workflow without putting in place adequate security controls.
- Insider threats can be exacerbated by the use of AI tools, particularly if access is not properly restricted.
Source: CyberScoop