Tenant Isolation for Multi-Tenant AI Products: RAG, Memory and Agent Tools
In a multi-tenant AI product, tenant data can reach the model through the retrieval index, agent memory, tool calls and caches. A prompt that says 'only use this customer's data' is not a boundary. Here are the four leak surfaces, the vector-store and database options for isolating them, and the checks that prove the boundary holds.
The central rule of multi-tenant AI is that the tenant boundary must be enforced outside the model, in code and in the datastore, never in the prompt. OWASP's 2025 Top 10 for LLM Applications names the risk directly: in multi-tenant environments where different groups of users share the same vector database, there is a risk of context leakage between users or queries, and OWASP's recommended mitigation is a permission-aware vector store with strict logical and access partitioning of datasets. This piece is our engineering analysis of where that boundary has to be enforced in an AI product (retrieval, memory, tools and caches), and of the trade-offs between the isolation options. It is an architecture opinion, not a description of any client system.
Why the Prompt Cannot Be the Boundary
The tempting design is to put the tenant in the system prompt: "You are answering for Acme. Only use Acme's documents." That is an instruction to a model, and a model can be steered. Prompt injection, instructions hidden in a document, an email or a tool result the model reads, can override or ignore it, and OWASP lists prompt injection as the first risk on the same list. We cover the mechanics in our prompt injection defence guide.
The engineering consequence is simple to state and easy to get wrong. The tenant identifier must come from the authenticated session, and it must be applied by your code on every data access. The model must never supply it, choose it or be able to change it. If a tool takes a tenant_id argument the model fills in, then the model is deciding whose data to read.
The Four Leak Surfaces
This is our framing. The surfaces are the places where one tenant's data can enter another tenant's model context.
The retrieval index
Embeddings and chunks from every tenant live in one or more vector stores. A query without a tenant restriction can return another tenant's chunks, and the model will happily use them. This is the surface OWASP's LLM08 describes.
Agent memory
Long-term memory (episodic, semantic or procedural) persists across sessions. If memory is keyed by user or agent rather than by tenant and user, one tenant's facts can be recalled into another's session. Our agent memory architecture guide covers the memory types.
Tool calls and credentials
An agent that calls internal APIs does so with some credential. If the credential is broader than the current tenant, any steered step can reach other tenants' records. This is the credential-scoping problem in our blast-radius analysis: the tenant scope belongs in a short-lived, narrow credential issued per request, not in the agent's good behaviour.
Caches, logs and evaluation sets
A semantic cache that matches on meaning can serve tenant A's cached answer to tenant B's similar question unless the cache key includes the tenant. Traces, prompt logs and golden datasets built from production traffic carry tenant data into places with different access rules.
Vector-Store Isolation Options
The "what the vendor documentation says" column is taken from the Qdrant and Weaviate documentation pages we opened on 16 November 2026. The "our assessment" column is our own reading. Other vector databases differ, so check the documentation for yours.
| Approach | What the vendor documentation says | Our assessment |
|---|---|---|
| Separate collection or index per tenant | Qdrant: rarely the most efficient approach, because each collection carries its own overhead, and only worth it for a limited number of tenants that need strict isolation. Qdrant Cloud's default limit is 1,000 collections per cluster. | Strongest separation of data, highest overhead. Suits few, large, high-sensitivity tenants. |
| Shared collection with a tenant filter (partition by payload) | Qdrant: efficient for a large number of small, similarly sized tenants; each point carries a tenant field and queries filter on it. | Cheap and scalable. The isolation is only as good as the filter, so every query path must apply it. |
| Dedicated shard per tenant | Qdrant: user-defined sharding gives each tenant its own shard, trading some resource overhead for stronger isolation. Weaviate: multi-tenancy stores each tenant on a separate shard, and it is disabled by default. | Stronger isolation than a filter without the cost of a collection per tenant. Needs capacity planning as tenant counts grow. |
| Tiered | Qdrant: small tenants share a shard; large tenants are promoted to their own. | A practical default when tenant sizes vary widely. |
The Relational Side: Row-Level Security
Most AI products also keep tenant data in a relational database that tools read from. PostgreSQL's row security feature is the standard way to enforce tenancy there, and its documentation is specific about the traps. As of the PostgreSQL 18 documentation (read 16 November 2026):
- When row security is enabled on a table, all normal access must be allowed by a policy, and if no policy exists a default-deny policy applies, so no rows are visible or modifiable.
- The table owner is typically not subject to the policies, although an owner can choose to be with
ALTER TABLE ... FORCE ROW LEVEL SECURITY. Superusers and roles with theBYPASSRLSattribute always bypass row security. - Referential-integrity checks, such as unique and foreign-key constraints, always bypass row security, and the documentation warns that schemas and policies must be designed to avoid covert-channel leaks through them.
The practical reading is that your application should connect as a role that does not own the tables and does not have BYPASSRLS, and that the tenant should be set on the connection or transaction from the authenticated session, so the database enforces the boundary even if application code forgets a WHERE clause. Treat any migration, admin or analytics role that bypasses row security as a privileged path with its own controls.
A Tenant-Scoped Retrieval Tool
from dataclasses import dataclass
@dataclass(frozen=True)
class RequestContext:
tenant_id: str # set from the verified session / token, never from model output
user_id: str
def make_search_tool(ctx: RequestContext):
"""Build a retrieval tool bound to one tenant for the lifetime of one request."""
def search_documents(query: str, top_k: int = 5) -> list[dict]:
# The model controls `query` and `top_k`. It does NOT control the tenant.
return vector_store.search(
query=query,
top_k=min(top_k, 20),
filter={"tenant_id": ctx.tenant_id}, # applied in code on every call
)
return search_documents # registered with the agent for this request onlyIllustrative and vendor-neutral; vector_store.search stands for your store's query call and its filter syntax will differ. The design point is that the tenant comes from the authenticated request context, and the tool's schema exposes no tenant parameter to the model.
The Database Enforces It Too
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
ALTER TABLE documents FORCE ROW LEVEL SECURITY; -- apply policies to the owner too
CREATE POLICY tenant_isolation ON documents
USING (tenant_id = current_setting('app.tenant_id')::uuid);
-- Per request, in the same transaction as the queries:
-- SET LOCAL app.tenant_id = '<tenant id from the verified session>';
-- The application connects as a role that does not own the table and has no BYPASSRLS.Illustrative PostgreSQL; adapt names and the way you set per-request settings. It shows the shape described in the PostgreSQL row-security documentation: enable row security, force it for the table owner, and write a policy keyed to a per-request setting.
A filter is only a guarantee if every code path applies it. The usual way a multi-tenant boundary fails is not the main request path but the side doors: an admin tool, a background re-indexing job, an evaluation script, a notebook with a service credential, a new tool added by someone who did not know about the convention. Isolation also has to be proved, not assumed: seed distinguishable test data in two tenants and assert that queries, memory recalls, cache lookups and tool calls made as one tenant never return the other's records. Isolation controls reduce one class of failure; they are not by themselves a compliance position, which depends on your contracts and the law that applies to you.
A Checklist for Tenant Isolation in AI Products
- Take the tenant identifier from the verified session and attach it in code on every retrieval, memory, tool and cache call. Never let a model-supplied argument choose it.
- Pick the vector-store isolation level deliberately (filter, shard or collection per tenant) and write down why. Check the limits in your store's documentation.
- Use database row-level security where tools read relational data, connect as a role that cannot bypass it, and review every role that can.
- Key agent memory by tenant and user, and include the tenant in every cache key, including semantic caches.
- Issue per-request, tenant-scoped, short-lived credentials to tools instead of a shared service credential.
- Decide who may see traces, prompt logs and evaluation datasets, and keep tenant identifiers in them so access can be controlled and deletion can be done.
- Add negative tests with seeded canary data in two tenants and run them in CI, including for background jobs and new tools.
- Plan tenant offboarding: be able to find and delete one tenant's embeddings, memory, caches and logs.
How Inductivee Intelligence Approaches This
This section is a design position, not a description of a client engagement. We do not publish client case studies; see our transparency page. When we scope an agentic SaaS product, we treat the tenant boundary as an architectural decision made before the first agent is wired up, in the same way that our AI-first SaaS patterns treat the core value loop. Our default is to enforce the boundary at the lowest layer that can enforce it (the database and the vector store), to give each agent role a narrow, per-request credential, and to write the negative isolation tests before the features they protect.
Sources (as of 16 November 2026)
Each page was opened on 16 November 2026. Vendor documentation changes, so check the current page before relying on a limit or a default.
Frequently Asked Questions
What is tenant isolation in an AI product?
Can a system prompt keep tenants separate?
Should each tenant get its own vector collection?
Does Postgres row-level security protect an AI app by itself?
How do you test that isolation works?
Written By
Inductivee Intelligence Team
AuthorAgentic AI Engineering Team
The Inductivee Intelligence engineering team — multi-agent orchestration, RAG pipeline architecture, and data liquidity engineering. Our writing is grounded in what we actually build, break, and operate in production.
Inductivee Intelligence is a remote-first agentic AI engineering studio. Our engineering content is written by active practitioners and technically reviewed before publication.
Engineer This With Inductivee Intelligence
The engineering patterns in this article are what our team builds into production every day. Explore the related service to see how we deliver this capability at enterprise scale.
Agentic Custom Software Engineering
We engineer autonomous agentic systems that orchestrate enterprise workflows and unlock the hidden liquidity of your proprietary data.
ServiceAutonomous Agentic SaaS
Agentic SaaS development and autonomous platform engineering — we build SaaS products whose core loop is powered by LangGraph and CrewAI agents that execute workflows, not just manage them.
Related Articles
Engineering AI-First SaaS: Architecture Patterns for Autonomous Product Features
Zero Credentials in the Sandbox: What Google's Agent Compute Engine Changes About Agent Blast Radius
Prompt Injection: A Defense-in-Depth Guide to Securing Enterprise LLM Applications
AI Agent Memory Architecture: Long-Term, Persistent Cognition for Production Agents
RAG Pipeline Architecture for the Enterprise: Five Layers Beyond the Basic Chatbot
Ready to Build This Into Your Enterprise?
Inductivee Intelligence engineers agentic systems, RAG pipelines, and enterprise data liquidity solutions. Let's scope your project.
Start a Project