Introduction to Agentic AI Security Risks
Context is the central plank of AI in general, and agentic AI in particular. If an AI system doesn’t have the correct context, it cannot make the correct decisions. Security is moving toward reliance on the autonomous and automatic action of agentic AI.
Emanuel Salmona, CEO at Nagomi Security, explains that an agent is only as good as the context it operates on. Give it an accurate, correlated view of your environment, and it can make decisions that genuinely reduce risk. Give it incomplete data, and it will still act, but confidently, quickly, and incorrectly.
Context is King
Context is of little relevance to Large Language Models (LLMs). Context here is fundamentally the user’s prompt – to which the LLM responds in accordance with its training. The LLM’s context is this prompt window, comprising both query and response; and it is stateless.
Agentic AI has a goal. Its context is stateful and includes anything and everything it is allowed to see and use to achieve its goal. If the context it is given does not include the relevance of a specific device to business continuity, the response it provides will not take that into consideration – it could make immediate shutdown its conclusion, unaware of the catastrophic business effect of shutting down that device at this moment.
The Problem of Inadequate Context
Too much context for an agent is similar to sensory overload for a human: slower reasoning and degraded performance, goal drift and loss of focus, oscillation between incompatible actions, and potential hallucinations as it attempts to connect loosely related bits of data.
Too little context is even more problematic. Just as humans might guess the answer to a problem by assuming bits of data that seem logical, so an agent that is instructed to achieve a goal might invent data to bridge the gap in its contextual knowledge. Operational accuracy and reliability may be lost through more hallucination.
Agentic AI in Security
Using AI to automate the work of the Security Operations Center (SOC) provides an example of potential agentic issues. The primary purpose of the original SOC is to manually triage alerts and find and respond to those that are most urgent and dangerous to the business and its IT infrastructure.
The difficulty for agentic AI in security is twofold. Firstly, it can only operate within the data it is given (which is its context). The conclusion it reaches while analyzing an alert within the confines of its context is entirely dependent on the adequacy of that context.
Adam Irwin, managing partner at Heligan Strategic Advisory, notes that no board would accept a set of numbers without an audit trail, yet many accept intelligence that shapes approvals and decisions with no method of visibility.
Alternative Approaches
Obbe Knoop, founder and CEO at Lanxit, has a different approach – his Security Decision Intelligence Layer uses artificial intelligence, but is not an agentic AI system. He believes that agentic AI is not sufficiently mature to be trusted with autonomous action; decisions and actions should currently be left in the hands of human experts.
Knoop gathers the context, fresh every time at the time of use. His product analyzes alerts in that current context and makes a recommendation within minutes. But it doesn’t simply say, this is critical or this is not critical; it explains why it has made its conclusion, and what the user should do about the situation.
Current State of AI Decision-Making
Our descriptions here are simplified, while AI and its use is evolving rapidly. LLMs are being given short-term memories, so they can have their own (limited) context, if only for the current session. Agentic AI concepts are also advancing in better context gathering, better decisions and usage with fewer hallucinations.
Accurate and relevant context is the axis upon which all else revolves. AI has been around for many years; but the current state of accessible AI is only a few years old. We should not expect it to behave as a mature technology, and yet we do.
What is already clear, however, is the current state of agentic AI can offer huge benefits or surprising failures depending on how we develop and manage it. Getting the context within which it operates is essential for beneficial performance. This is possible, but as we have seen, it is very difficult to achieve because of all the pitfalls discussed above.
Source: SecurityWeek