The cache is customer data too.

It is also a copy of your prompt, in another form. Not readable text, but a mathematical representation derived from it, held in memory for minutes or hours. For a firm that has promised a client that documents are processed and then forgotten, that is worth knowing.

Precision matters, so here are the limits. The cross-customer sharing that audit found was mostly at smaller open-model hosts, plus one OpenAI embeddings endpoint. The authors could not extract anyone's prompts in practice, and at least five providers changed their systems after disclosure. Nobody has shown one customer reading another's prompts at Anthropic, OpenAI, Google or AWS.

That is good engineering, and it answers the question most vendor reviews ask. It leaves the question they rarely ask: what about users inside the same customer? AWS says plainly that within one account, cache entries might be shared across requests, and suggests adding a tenant identifier to separate them. A 2026 workshop paper found that an API gateway's shared-credential mode broke providers' account isolation entirely, until customers brought their own keys.

If you run models yourself, the default matters more. vLLM, a widely used open-source server, enables prefix caching by default and separates users only when each request carries a cache_salt. Its own documentation describes that salt as the protection against timing attacks. Without it, the cache is shared by every user of the server.

The fix is not exotic. Scope caches to the same boundary as document access: per matter, per deal team, per client, or per user where the stakes justify the lost speed. Treat the cache salt or tenant key as an access-control decision, owned by the people who own the barrier, not as a performance setting owned by whoever deploys the server.

The regulation defines personal data as "any information relating to an identified or identifiable natural person", and processing as including storage, retrieval and erasure. A cache derived from a prompt about a named client is, on that reading, personal data held for a period. Storage limitation requires it be kept "for no longer than is necessary". The erasure right requires deletion "without undue delay". Article 28 requires a processor to delete personal data and "existing copies" at the end of the service. And Article 32 requires security "appropriate to the risk", which a known side channel is part of.

Most of this is easy to satisfy, because caches are short-lived by design. But lifetimes differ: from five minutes to 24 hours depending on provider and setting, and OpenAI notes that manual cache clearing is not currently available. A firm that promises erasure should know which of those it is running, and write it down.

The cache is a performance feature. It is also customer data, in another form, for a while. Both things are true, and only one of them usually gets written down.