Semantic caching: test incorrect cross-user reuse

A semantic cache can save time and cost by reusing an answer for a similar request. Similar wording does not imply equal authority, tenant, policy version or source visibility. A cache key that ignores those boundaries can turn optimization into data exposure.

Deny unsafe reuse

The LLMOps and security lab stores one synthetic answer for tenant T1, a standard user and policy v3. Only the request with the same boundary fields hits.

from llmops_cases import semantic_cache_case

result = semantic_cache_case()
assert result["hit_indices"] == [0]
assert result["requests"][1]["cache_hit"] is False
assert result["requests"][2]["cache_hit"] is False
assert result["requests"][3]["cache_hit"] is False
assert result["semantic_similarity_alone_used"] is False

The rejected requests change tenant, permission scope or policy version. The fixture does not run embeddings or measure cache quality.

Build the key from the decision context

Partition by tenant and authorization class before any similarity lookup. Include application, prompt/model, retrieval index, policy and tool-contract versions when they can change the answer. Add locale, output schema and relevant feature flags. If a response contains user-specific or time-sensitive data, disable shared caching or scope it to that user and a short validity window.

Store provenance with the cached value: sources, creation time, expiration, eligibility filters and the acceptance result. Revalidate permissions and source freshness on retrieval. Encrypt stored values, restrict cache access and avoid using raw personal text as a key or metric label.

Test negative pairs

Positive tests show that reuse works. Security comes from negative pairs with nearly identical wording and one changed boundary: different tenant, role, geography, policy version, source entitlement or deleted record. Assert a miss before inspecting the cached payload. Add deletion tests so revoked content cannot survive in an old answer.

Measure hit rate only after cross-boundary reuse remains zero in the tested set. A higher hit rate is not an improvement if it weakens isolation.

The Generative & Agentic AI course connects caching to tenant authorization, RAG freshness and release versioning.

Reproducible exercise

Create twenty paired requests that differ in one security-relevant field. Show that a text-only key causes false hits, then replace it with an eligibility partition plus semantic lookup and rerun every negative test.

Continue learning

This article is part of the LLMOps, security and operational evaluation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Design model fallback without silently lowering acceptance criteria.
• Continue with Set per-user quotas for expensive AI operations.

Reference: OWASP LLM02:2025 Sensitive Information Disclosure.