Security Reviews of Enterprise AI Systems in 2026
Lessons from Recent Disclosures and Real-World Reviews
When I look at design docs for RAG and MCP systems, the security section is usually the same few lines. Retrieval so the model doesn't hallucinate. Tools so it can act. A prompt-injection classifier on the way out.
The review often stops there, as if this were still a chatbot. In practice these systems read untrusted text, search a corpus that no longer carries the original file permissions, and call tools as a service account.
This post is about how that architecture actually moves data and credentials, two incidents that already showed what goes wrong, and the questions worth asking in a review.
How the system actually works
A production RAG path is more than "the model plus some documents."
Source text is chunked, sent to an embedding API, stored in a vector database, retrieved by similarity, stuffed into a prompt, and often sent on to a foundation-model provider. Every hop is another place the data lives.
Treat the vector database like the document store. Embeddings are not an anonymized form of the corpus. Morris, Kuleshov, Shmatikov, and Rush showed that Vec2Text could recover 92% of 32-token inputs exactly, and 89% of full names from embedded MIMIC clinical notes.
MCP adds a second path: tools running in a host such as Cursor, Claude Desktop, or your orchestrator.
The MCP authorization spec (2026-07-28) is clear on identity. Authorization is optional. HTTP servers should use OAuth. Stdio servers should not; they take credentials from the environment instead.
When HTTP OAuth exists, it only controls access to the MCP server. It does not map each tool call onto the signed-in user's role. If that server then calls GitHub or SharePoint, the spec forbids token passthrough. The upstream call uses a separate token, and that token does not have to be the user's least-privilege role.
Three identities matter here: the user, the MCP client (the host app), and the MCP server plus whatever it calls next. Ask which one is actually talking to GitHub, SharePoint, or the database. If the answer is a bot PAT in the host environment, the user's IdP group is not doing anything useful.
Anthropic's connector docs say Claude acts "based on your permissions." That is a product claim for their OAuth flow, not a guarantee of the protocol. The same help article still tells you to treat custom MCP servers as a prompt-injection risk.
The harness will change underneath you
A lot of the agents I see in reviews sit on Claude Code, Cursor, Codex, or whatever the company standardized on. Those products look like a platform, but they behave more like a research harness wrapped in a CLI.
Boris Cherny, who built Claude Code, talked about this in a recent Y Combinator Startup School interview. On every new model they ablate the product: delete the system prompt, bring it back line by line, unship tools, delete harness code. For Opus 5 they removed about 80% of the system prompt, because the new model no longer needed the old corrections. The interviewer described it as starting from scratch. Cherny's correction was they don't wipe the whole codebase. They still delete a lot, because each model is different enough that last quarter's prompt may not apply.
That is a reasonable way to build a frontier coding agent. It is a problem for anyone who treated last quarter's harness as a stable control. Permission copy, tool lists, and "the agent won't do X" instructions live inside something you don't own and that the vendor will keep rewriting. A more capable model can also do more with the same PAT and the same MCP servers you already connected. You don't get a design review when that happens. You get a changelog.
If the internal agent is built on one of these foundations, put authorization, ACLs, spend caps, and write-approval outside the harness. Re-review when the model or the CLI moves. A screenshot of last quarter's confirmation UI doesn't tell you this quarter's harness still asks.
Two incidents, same architecture
In May 2025, Invariant Labs showed this against the official GitHub MCP server.
Someone filed a malicious issue on a public repo. A user asked Claude 4 Opus to look at the issues. The agent used the victim's GitHub PAT, then opened a public pull request with private repo names, relocation plans, and salary. Claude Desktop did ask for confirmation. A lot of people had already clicked Always Allow.
Invariant's point was that this came from the architecture, not a bug in the GitHub MCP server: trusted tools, untrusted content, and a way to send data out, all in one session.
Simon Willison called that pattern a lethal trifecta: private data, untrusted content, and external communication. Guardrail vendors often quote a ~95% catch rate. As Willison notes, in web application security that is a failing grade. A vendor product will not save a team that wired all three into one agent.
Microsoft 365 Copilot had the RAG-shaped version. EchoLeak (CVE-2025-32711) was a zero-click prompt injection in a production system. A crafted email landed in the retrieval index. First-person phrasing slipped past XPIA classifiers. Markdown and auto-fetched image URLs carried the data out, through a Teams proxy that CSP allowed.
The user never pasted a jailbreak. Retrieval did the work. Images, links, and citations were the tools.
Start the review with identity and data flow, not with a jailbreak classifier.
What shows up in review
These are failure modes, not one-off CVEs. A system that "uses Azure AI Search" and "uses MCP" can still hit all of them.
The index is an unauthenticated database with a search box
Document ACLs on Confluence or SharePoint do not automatically apply to chunks. If retrieval is only "find similar text," that is an authorization bypass.
OWASP's RAG cheat sheet calls this the most common compliance failure in enterprise RAG deployments. Store ACL metadata on every chunk. Enforce it at retrieval time. Do not ask the language model who is allowed to see a passage.
Filter before you retrieve. If you fetch first and filter after, similarity scores of documents the caller cannot read still leak. When a SharePoint file is revoked, delete the chunks too, or it lives in the index forever.
Azure AI Search document-level access control (updated 8 August 2026) is Microsoft's version of this. Security-filter string matching is generally available. Native POSIX ACL / RBAC, Purview labels, and SharePoint ACL paths are still preview, and there is a documented lag before the preview API notices permission changes.
"We use Azure AI Search" is not the same as "we trim results using the caller's token." Microsoft's GPT-RAG sample is the pattern that works: one token for the orchestrator, a separate on-behalf-of token for Search. Two audiences. Not interchangeable.
Vector APIs often ship with authentication turned off. CVE-2026-45829 in ChromaDB is what that looks like when a collections endpoint is on the network. Bind HTTP MCP to localhost. Don't put it on 0.0.0.0 without a token.
Give embeddings the same classification as the source text. Encrypt them at rest. Don't expose a public embedding API. Rate-limit anything that can be queried as an oracle. Anderson et al. showed you can ask a RAG system, in ordinary language, whether a passage is in the index.
Retrieval is a prompt-injection primitive
RAG does not fix prompt injection. It moves it into the corpus.
Greshake et al. (2023) put it well: these apps blur data and instructions, and retrieved text can decide which APIs get called. OWASP LLM01:2026 has the same RAG scenario: an attacker edits a document in the retrieval repository. Delimiters and "this is data, not commands" help, but they don't create a trust boundary.
Poisoning also doesn't have to look like ignore previous instructions. PoisonedRAG (USENIX Security 2025) got a 90% attack success rate by injecting five attacker-chosen texts per target question into a knowledge base with millions of documents. The texts were written to be retrieved, and then to steer the answer. Scanning the corpus for jailbreak strings will miss that.
If the agent can write back (tickets, PRs, wiki pages, "memory," a vector upsert), one injection becomes every future retrieval. The OWASP cheat sheet says no application code or agent endpoint should have direct write access to the vector index. Ingest with a locked pipeline identity. Put a human on writes. GitHub MCP's public PR was write-back used as exfil, and that write then becomes someone else's RAG context.
Private data, untrusted content, and an egress channel
You don't need a compromised MCP server. You need all three in one session.
Untrusted content is issues, email, web pages, RAG chunks, and tool descriptions. Privileged tools are PATs, send-mail, create-PR, fetch_url. Egress is a public PR, an outbound image URL, a search query, an email subject, or a hidden sidenote argument.
OWASP LLM03:2026 (Excessive Agency) is the mail-summarizer story. An injection in an email tells the agent to scan the inbox and forward mail to the attacker. You avoid that with read-only OAuth, no send function, and a human in the loop before send. Rate-limiting send limits the damage. It does not prevent it. For GitHub, the same cut is repo-scoped tokens and one untrusted source per session.
A confirmation UI that only shows the tool name misses the arguments that matter. The MCP cheat sheet wants full parameters. Invariant's tool-poisoning writeup is the reason: Cursor hid the argument that was an SSH key. Always Allow undoes the rest.
MCP identity is not the user's role
There are three ways this usually gets built.
Passthrough reuses the inbound token. The spec bans it, because audience binding and audit fall apart. A shared service identity ignores the user, so every prompt-injected caller inherits the bot. On-behalf-of mints a new token for a new audience, still representing the user. Only that last option, plus query-time ACL, actually carries user authorization into retrieval and tools.
Default scopes are easy to get wrong. If the 401 WWW-Authenticate challenge leaves out scope, a spec-following client asks for every scope in scopes_supported. Publish files:*, admin, and db:*, and the client will request the lot on first handshake. Least privilege in the spec is SHOULD, not MUST. You still need authorization on the server even when the token lists scopes.
Confused deputy shows up twice. In the spec, it is an MCP gateway that OAuths to Slack or Google with one static client_id, then skips consent for a later malicious client. In OWASP, it is an extension that says it reads "the current user's documents" while connecting as an account that can see everyone. Same name, two mechanics. Both belong in the same review.
State handles (a cart id, a workflow id) are not authentication. MCP servers mint them as tool arguments. Holding the id is not proof of who you are. Bind them on the server to the user on the token. Don't share MCP connections across users.
SSRF is in the spec too, during OAuth discovery. Clients fetch resource_metadata and token endpoints. A remote MCP URL is an SSRF primitive against the host, including cloud metadata and localhost. Tools that fetch URLs from model-generated parameters have the same problem.
The plugin is the supply chain and the prompt
Tool descriptions sit in the model context. MCP assumes they are harmless.
Invariant's tool-poisoning attacks (April 2025) hid instructions in an add tool: read mcp.json and id_rsa, pass them as sidenote. A rug pull is when the description changes after you approved it. Shadowing is when a malicious add retargets a trusted send_email. WhatsApp MCP was the same idea against a trusted messenger: encrypting the chat does not protect the agent.
Pin tool schema hashes. Show full descriptions in the consent UI. Keep untrusted marketplace servers away from secrets. RAG ingest connectors (Drive, SharePoint, scrapers) are the corpus supply chain. Pin embedding-model versions and stage them before they hit the index. A "verified" marketplace badge is marketing unless there is signing and a hash pin. OWASP LLM04:2026 is the name for this. The control is the pin.
Boring controls missing on an unbounded loop
Rate limits and spend caps stop people probing the corpus, testing membership, flooding retrieval, and looping the agent. OWASP moved Unbounded Consumption to LLM06:2026 for a reason. Token-metered APIs plus tool loops are a way to run up the bill. Limit tokens per request, not just requests per second.
Logs have to survive the injection. After the fact you need who queried, which chunks came back, which tool ran, and with which arguments. Token passthrough destroys that trail.
Full-fidelity traces are a second copy of the corpus, plus secrets. Who can read LangSmith or App Insights is a data-residency question, not a debug flag. Retrieved chunks travel with the prompt to the cloud. OWASP AI Exchange is explicit about that.
Output handling is LLM10:2026. Model output becomes the next tool argument, a markdown image, or HTML in a chat UI. Constrain tool arguments to a schema. NVIDIA NeMo Guardrails can reject or mask a chunk before it enters the prompt. That is the right layer, even if a jailbreak detector is incomplete.
Fail closed. If the ACL check errors, don't fall back to "just the model."
One thing worth calling out: "we enabled guardrails" is not enough. Ask which tools sit outside the pipeline they wrapped.
Showing sources is not the same as access control
OWASP lists RAG as a way to reduce misinformation (LLM07:2026) if you retrieve from a trusted database. PoisonedRAG assumes the database is not trusted. A poisoned chunk will still be cited. People, and other agents, treat that citation as proof.
Groundedness doesn't decide who is allowed to see a document. Showing sources doesn't replace access control.
Classifiers that look for "you are Copilot" miss first-person EchoLeak phrasing. Model alignment did not stop Claude 4 Opus on GitHub MCP. If retrieval fails and the system still answers, that is fail-open.
What the review has to force into the design
Write two identity decisions down so they can be tested.
# Mutable identity: stdio MCP runs as the host environment.
# Every prompt-injected user inherits this PAT.
mcpServers:
github:
command: npx
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: ${GITHUB_PAT}
# Immutable identity: a new token, new audience, this user, this session.
# GPT-RAG's OBO into Azure AI Search is the same shape:
# orchestrator token ≠ Search token; they are not interchangeable.
# Similarity-only. Source ACLs died at ingest.
hits = store.query(q_emb, k=5)
# Query-time filter bound to the caller. Still wrong if ACL metadata is
# stale or you filter after retrieval, but this is the control OWASP names.
hits = store.query(q_emb, k=5, filter={"acl": {"$contains": user.oid}})
Then split the tools. Read mail without send. Retrieve without upsert. One repo per session. Don't ship run_shell if a narrower tool would do. Put a human on writes, with the arguments visible. Pin the MCP server and the tool schemas. Cap how much you retrieve. Log the pipeline. Fail closed.
OWASP is useful as a shared vocabulary. LLM Top 10 2026 (published 4 August 2026) put Excessive Agency at LLM03, Unbounded Consumption at LLM06, Vector and Embedding Weaknesses at LLM09, and Improper Output Handling at LLM10. The RAG and MCP cheat sheets are more practical for a review. The Agentic Top 10 2026 names help once you have a data-flow diagram.
Putting it together
An enterprise AI system still needs the same questions you'd ask of any service that reads customer data and calls GitHub as a bot.
Whose token is on the wire? Who can write the corpus? What can the agent do after it has read untrusted text? What happens when the ACL check fails? What happens when the vendor ships a new model and deletes half the harness you reviewed?
Key takeaways
Start with identity and data flow, not with a prompt-injection classifier.
Enforce access control at query time, on the caller's token. Source-system permissions do not follow chunks. Embeddings are the documents.
Treat retrieved content as untrusted input. Who can write the corpus matters as much as who can query it. Agents should not write into the index they read.
Don't put private data, untrusted content, and a way to send data out in the same session. Human approval has to show full tool arguments. Always Allow undoes it.
MCP OAuth is optional, stdio uses environment credentials, and passthrough is forbidden. Ask which identity is on the wire, and prefer on-behalf-of plus query-time ACL over a shared bot PAT.
Tool descriptions are in the prompt. Pin schemas, show them in full, and treat ingest connectors as supply chain.
Rate-limit, cap spend, log the pipeline, and fail closed on ACL errors. Citations and model alignment don't replace authorization.
If the agent sits on Claude Code, Cursor, or a similar harness, assume prompts and tools will change on the next model. Keep authorization outside the harness and re-review on upgrades.
References
MCP Authorization (2026-07-28) and Authorization Security Considerations
Azure AI Search: Document-Level Access Control (updated 2026-08-08)
Boris Cherny, Building Claude Code (Y Combinator Startup School interview)
Greshake et al., Not what you've signed up for (2023)
Zou, Geng, Wang, Jia, PoisonedRAG (USENIX Security 2025)
Morris et al., Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)
Reddy & Gujral, EchoLeak (AAAI Symposium); CVE-2025-32711
Invariant Labs: GitHub MCP Exploited (May 2025)
Invariant Labs: Tool Poisoning Attacks (April 2025)
Anderson et al., Is My Data in Your Retrieval Database?
CVE-2026-45829 (ChromaDB)
Simon Willison, The lethal trifecta (16 June 2025)
