Artificial intelligence is rapidly moving beyond systems that simply generate text, images or predictions. Modern AI systems can retrieve organisational information, maintain memory, call APIs, use software tools, interact with other systems and increasingly perform tasks with limited human intervention.
This transition from AI models to AI agents fundamentally changes the cybersecurity landscape.
Who requested the action?
What information influenced the AI?
What is the AI allowed to do?
Can the action cause harm?
Can we reconstruct what happened afterwards?
The Expanding AI Attack Surface
A standalone language model has a relatively limited attack surface. An enterprise AI system may connect users, models, organisational data, memory, RAG systems, agents, APIs, software tools and external services.
An AI agent may be able to:
- Search organisational documents
- Read and process email
- Query databases
- Access cloud storage
- Generate or execute code
- Call internal and external APIs
- Create or modify files
- Communicate with other agents
- Trigger business processes
It becomes: “What can an attacker cause the AI to do?”
1. Protect Identity and Intent
Every significant AI operation should have a traceable identity. Security controls should establish who the user is, which AI agent is acting, on whose authority it is operating and what task has been authorised.
An authenticated user should not automatically give an AI agent unrestricted access to everything available to that user.
The agent should operate within the minimum permissions necessary to complete the requested task. This extends the principle of least privilege into AI environments.
2. Treat Context as a Security Boundary
AI decisions are heavily influenced by context. This context may come from user prompts, previous conversations, retrieved documents, databases, websites, email, vector stores, memory systems or other agents.
Not all context should be trusted equally.
A legitimate user may ask an AI assistant to summarise a document. The document itself could contain malicious instructions intended for the AI rather than the human reader.
Security systems therefore need to evaluate both who requested an operation and what information influenced the AI’s decision.
3. Control AI Autonomy
The risk of an AI system increases significantly when it can take actions. Generating an incorrect answer is one type of failure. Deleting a file, changing a configuration, executing code or transmitting confidential information is another.
Organisations should explicitly define what an AI agent can:
Low-risk actions may execute automatically. Medium-risk actions may require additional verification. High-risk or irreversible actions should require explicit human approval.
4. Apply Least Privilege to Agents and Tools
AI agents should not receive permanent broad credentials simply because they may occasionally require access to a resource.
Where possible, permissions should be:
- Task-specific
- Time-limited
- Resource-limited
- Context-dependent
- Revocable
- Auditable
Tools should also be individually authorised. An agent that needs to search a database may require read access but not permission to modify or delete records.
5. Secure RAG, Knowledge and AI Memory
Retrieval-Augmented Generation allows AI systems to use organisational knowledge without retraining the underlying model, but it introduces additional security risks.
Attackers may attempt to insert malicious information into knowledge sources, manipulate retrieved content, expose restricted documents or influence AI decisions through poisoned data.
AI knowledge should be treated as a security-controlled information asset rather than simply additional model context.
6. Defend Against Prompt Injection
Prompt injection should be considered a fundamental AI attack vector.
Direct prompt injection occurs when an attacker intentionally provides malicious instructions to the AI. Indirect prompt injection may originate from:
- Web pages
- PDF files
- Documents
- Source code
- Database records
- API responses
- RAG knowledge
- Messages from other AI agents
Input validation + Context isolation + Least privilege + Tool restrictions + Output validation + Action approval + Monitoring
A useful security principle is: External content is data, not authority.
7. Introduce Runtime AI Security
Agentic AI requires security decisions while the system is operating. Before executing a sensitive action, the security layer should evaluate:
The security decision may therefore be more sophisticated than simply ALLOW or DENY.
8. Make AI Activity Observable
AI systems should not become invisible decision-makers inside enterprise infrastructure.
Security teams need visibility across the complete AI activity chain:
Important security telemetry may include:
- User and agent identity
- Authentication events
- Prompt and context risk indicators
- Retrieved data sources
- Tool invocation
- API calls
- Sensitive-data access
- Permission changes
- External communication
- High-risk actions
The objective is to answer an important forensic question: “Why did the AI perform this action?”
9. Bring AI into the SOC
AI security should integrate with existing cybersecurity operations rather than operate separately.
Security teams should detect behaviours such as:
- Repeated prompt-injection attempts
- Abnormal tool invocation
- Unexpected sensitive-data retrieval
- Unusual external destinations
- Agent privilege escalation
- RAG manipulation
- Suspicious memory modification
- High-volume automated actions
- Agent-to-agent anomalies
10. Continuously Test AI Security
Traditional vulnerability scanning alone cannot adequately test systems whose behaviour depends on natural-language instructions, retrieved information, models, tools and changing context.
AI security testing should include:
- Direct prompt injection
- Indirect prompt injection
- Jailbreak attempts
- Sensitive-data extraction
- RAG poisoning
- Memory manipulation
- Tool abuse
- API misuse
- Privilege escalation
- Agent impersonation
- Autonomous-action abuse
- Agent-to-agent attacks
A Practical AI Security Framework
IDENTIFY
Identify AI models, agents, data sources, tools, APIs, identities, permissions and business processes.
CONTROL
Apply authentication, authorisation, data classification, contextual access control and least privilege.
ISOLATE
Separate untrusted content, tools, execution environments, credentials and sensitive resources.
MONITOR
Observe AI behaviour, tool use, data access, external communications and risk indicators.
RESPOND
Block dangerous actions, revoke permissions, isolate compromised agents, preserve evidence and investigate incidents.
TEST
Continuously red-team AI applications, agents, RAG systems, memory, tools and security controls.
From Model Security to AI System Security
The central shift in AI cybersecurity is recognising that the model is only one component of the system.
A highly secured model connected to unrestricted tools, excessive privileges, poisoned knowledge sources and poorly monitored APIs can still create a highly insecure environment.
↓
AI MODEL / AGENT
↓
DATA + RAG + MEMORY + TOOLS
↓
ACTION GUARDRAILS
↓
ENTERPRISE SYSTEMS
Access Control • Risk Assessment • Observability • Audit • Incident Response
The Security Principle for the Agentic Era
AI agents will increasingly operate inside organisations alongside humans, applications and machines. They will access information, make decisions and execute actions at machine speed.
Every consequential AI action should therefore be attributable, authorised, constrained and observable.
The future of AI security is not simply model security. It is AI system security — protecting the identities, data, context, memory, tools, decisions and actions surrounding increasingly autonomous intelligence.