One Model, Many Customers, and the Leak Nobody Tests For

A hotel floor looks like strong isolation. Every room has its own door, its own number, and its own guest, and nobody expects one room's contents to turn up next door. The failure that matters is rarely a door left open. It is a keycard cut for one room that quietly opens the one beside it. That is the shape of the risk in shared AI systems. The separation between customers looks structural until something built for efficiency, a shared model, a shared index, or a shared cache, turns into a keycard that opens more than one room.

A B2B software company ships an AI feature. It summarizes contracts, answers questions over a customer's documents, or drafts replies from a customer's history. The demo lands, the feature works, usage climbs. One test rarely makes the plan. Can customer A pull customer B's data out of the same system. The feature working and the feature keeping customers apart are two different claims, and only the first one shows up in a product demo.

The word tenant here means one customer's isolated slice of a shared system, the same way a hotel gives each guest a private room inside one shared building. Most B2B AI products are multi-tenant by design, because running a separate model, database, and server stack for every customer is expensive. The saving comes from sharing, and sharing is exactly where the boundary between tenants can quietly disappear. Three surfaces tend to be shared, the model weights, the retrieval layer, and the serving cache. Each one has a documented way of moving one tenant's data into another tenant's output.

The Model Can Memorize One Tenant and Recite It to Another

Language models memorize. One experiment extracted hundreds of verbatim sequences from a public model using only ordinary query access, including names, phone numbers, and other personal details, and some of those strings appeared in just a single training document ^[1]. The finding that should worry anyone pooling customer data is that larger models memorized more, not less. A string does not need to be common to be recoverable.

The multi-tenant version of this risk shows up when a single model is fine-tuned on the combined data of every customer. Fine-tuning means continuing to train a base model on your own examples so it adapts to your domain. Do it on the pooled corpus of all tenants and the weights themselves become a shared surface. A prompt from tenant B can, at least in principle, surface a string that only tenant A ever provided, and no request-time permission check sees it happen, because the leak is baked into the parameters rather than pulled from a database.

There is a genuinely interesting counterpoint here. One study found that retrieval-augmented generation (RAG) can reduce a model's tendency to reproduce memorized training data, because the model leans on the freshly retrieved text instead of its own parameters ^[2]. That is real, but it moves the risk rather than removing it. The retrieved text is the next shared surface.

Retrieval Ranks by Relevance, Not by Permission

Retrieval systems have one job, to find the most relevant text for a question, and they are good at it. Relevance is not permission. A baggage carousel sorts bags by the flight they arrived on and trusts travelers to claim the right one, which holds up until two bags look alike. A retrieval index sorts by semantic similarity, how close two pieces of text are in meaning, and by default carries no idea of who is allowed to see what. The index behind a shared RAG feature is usually one index for all tenants, kept that way for cost and for retrieval quality. The documents most similar to a question get pulled into the model's context before anything has checked whose documents they are.

One research group names this the relevance-authorization gap, and its measurements are stark. In a multitenant test setup, retrieval with no authorization gate leaked cross-tenant data in 98 to 100 percent of probes ^[3]. The proposed fix is not a smarter filter placed after retrieval runs. It is a layered architecture that gates the candidate set at ingestion and at retrieval time, before the ranking step ever touches a document the requesting tenant should not see.

A follow-up test used a composite structured prompt, one part written to trigger retrieval and one part instructing the model to repeat the context it was given, and pulled 112 exact matches and 208 near matches from 250 prompts against a private email dataset ^[2]. The common instinct, retrieve everything and then filter the results the user should not see, arrives too late. By the time the filter runs, the restricted text is already sitting in the context window.

The Cache Is a Side Channel

To avoid recomputing work, serving systems store intermediate state, often the key-value cache that holds a request's attention data, and reuse it when a new request shares a prefix with an earlier one. Shared across tenants, a cache hit comes back faster than a cache miss, the way a mailroom hands over a familiar package quicker than a first-time delivery, and that timing gap is measurable from the outside. A watchful neighbor can learn what someone nearby received without ever seeing it.

One attack, in work now published in IEEE Transactions on Information Forensics and Security, turned that difference into a working exploit. It detected per-token cache hits at 86 percent accuracy, recovered prompt tokens at 92.3 percent, and a peeping neighbor variant identified another user's cached content above 95 percent accuracy after three trials ^[4]. In another experiment, presented at the Network and Distributed System Security Symposium, the same idea ran against SGLang, a production serving framework, and reconstructed a stranger's entire prompt with no prior knowledge of its content at a 95 percent success rate, a figure that climbed to 99 percent once the attacker already knew the surrounding template. A case study inside that same paper recovered a health questionnaire's embedded gender, age, weight, and height fields in just 60 requests ^[5]. None of this requires special access. The attacker just sends normal requests and measures how long the answers take.

This is not confined to research prototypes. An audit of live commercial APIs, presented at ICML 2025, detected global cache sharing across users in seven providers, including OpenAI, which means a fast response can carry information about a stranger's prompt ^[6]. The optimization that makes shared inference affordable is the same one that opens the channel.

Where the Boundary Has to Live

The pattern across all three surfaces is the same. The boundary cannot be a filter at the end of the pipeline. It has to be enforced before the shared resource is touched. For retrieval, that means gating the candidate set before the model reads anything, not after, an approach validated earlier against ungated retrieval ^[3]. For the model, pooling every customer into one fine-tune trades isolation for convenience, and per-tenant routing or per-tenant adapters keep one customer's data out of the surface another customer queries. For the cache, the same tension holds. Per-tenant cache isolation closes the timing channel entirely, but it gives up the throughput gains that shared caching exists to provide, and that tradeoff has to be sized against how sensitive the cached content is rather than defaulted away in either direction. Isolation is not free, and its cost belongs in the design, not in the incident report.

This is the case for carrying tenant context, the identity of the customer a request belongs to, through every layer of the system. Retrieval, model selection, cache, and logging each need to know which tenant they are serving, rather than trusting a single check at the front door. A boundary enforced at each layer is harder to build than a filter bolted onto the output, and it is close to what confidential computing work already argues for at the hardware level, where isolation is treated as a property of the whole path rather than a wrapper around it ^[7]. The engineering effort is the difference between a feature that works and a feature that can prove it keeps customers apart. A platform should treat per-tenant isolation, model routing, and tenant context at every layer as the default rather than a retrofit, for exactly the reasons the research keeps surfacing. The related tradeoffs are discussed further in The Path to Practical Confidential Computing for AI Systems.

What to Actually Test

The useful test is adversarial and a little boring. Sit in tenant B and try to retrieve, extract, or time a path into tenant A's data. Ask the model to repeat the context it was handed. Send prompts that share long prefixes with a known tenant and watch the latency. Probe the retrieval index with a query only another tenant's documents could answer. The evidence across memorization, retrieval, and caching says these leaks are reachable with ordinary access, so the open question for any team shipping B2B AI is not whether the failure mode exists. It is how much isolation the product can afford while still hitting its latency and cost targets. That number is worth knowing before a customer's security review asks for it.