A lot of teams still talk about RAG as if retrieval is the security boundary. I do not think that holds up in practice. The real control points sit earlier and later than retrieval itself: before data enters the prompt and before the output is used.
Where the boundary actually breaks
Once data lands in the context window, the model does not preserve your neat mental categories. User prompt. System prompt. Retrieved document. Tool output. Conversation history. It all becomes one stream of tokens.
That is the flat namespace problem.
The context window has no internal permissions model. If sensitive and non-sensitive data enter the same prompt, the model does not enforce the boundary you were hoping existed.
That sounds obvious once stated plainly. It is still not how many systems are built.
I keep seeing architectures that assume the vector database or retrieval layer is doing access control for the model. The idea goes something like this: "we only retrieve the relevant documents, therefore the model only sees what it needs." But relevance is not the same thing as sensitivity, and retrieval is not the same thing as containment.
If your system retrieves a customer record, an internal note, and a prompt injection payload from a poisoned document, those tokens land in the same working space. The model does not stop and ask which part came from an authorised business process and which part came from hostile context. It just sees text.
That is why the OWASP GenAI Data Security Risks and Mitigations 2026 guide matters. It treats the context window as shared memory with no internal access controls and calls out over-broad context windows directly. The older LLM and Gen AI Data Security Best Practices was already pointing in the same direction, but the 2026 guide is blunter about where teams keep getting this wrong.
A generic example is enough to show the problem. Say you have a finance assistant connected to internal policy docs, board minutes, and customer support records. The user asks an innocent question. Retrieval pulls a few relevant snippets. One of those snippets includes instructions left inside a document or generated by another tool. Once all of that is in the same context, the access model you had in mind no longer maps cleanly to what the model sees. The question is no longer whether the vector store had categories. The question is what the model can now do with everything it sees together.
This gets worse when tools are chained.
Slack output enters the prompt. Then Jira output. Then an internal document. Then an email draft based on all of it. One tool's output becomes another step's input. The final model call inherits the whole mess unless you deliberately split the workflow.
So the practical controls are not mysterious.
- Classify before retrieval. Decide what data is allowed anywhere near the model in the first place. Some data should stay behind an API and only return a computed answer. Some should never enter a prompt at all.
- Separate sensitive steps into separate invocations. If the workflow needs HR data and customer data, that does not automatically mean one model call should see both. Split the task. Preserve boundaries deliberately.
- Sanitise anything that re-enters the context. Tool outputs are not trustworthy just because the tool is internal. Strip metadata, identifiers, and anything the next step does not need.
- Treat output use as another control point. The fact that the model produced a plausible answer does not mean it stayed within the data boundaries you intended.
This is where the compliance angle stops being abstract. Article 10 of the EU AI Act puts real weight behind data governance, provenance, and documented handling for high-risk systems. Even outside that regime, the underlying point stands: if you cannot explain what data entered the system, why it entered, and what downstream steps touched it, you do not have defensible control.
RAG is useful. Vector databases are useful. Neither of them is a magic privacy layer.
The model does not know that one paragraph was a harmless policy extract and the next one was a confidential note that should never have travelled that far. That distinction is your job, in architecture and workflow design, before the prompt is assembled.
The context window is memory, not a security boundary. Treat it like one and the risk of unintended disclosure rises quickly.
Related
- Securing Your LLM Applications with the OWASP Top 10 for the full OWASP LLM risk profile
- Adding AI to your SaaS for the broader risks of third-party AI solutions