Redis Launches LangCache, a Managed Semantic Cache for LLM Applications

Redis frames the target as the repetition that fills production LLM traffic. Support assistants and retrieval-augmented generation pipelines field the same intents thousands of times a day, each phrased differently, and most stacks bill every phrasing as a fresh request. Three requests to a customer-support assistant illustrate the gap: "Can I get a refund after buying the monthly plan?", "Is the monthly subscription refundable?" and "Can I cancel the plan and get my money back?" The wording differs, but the question and the answer are identical. Prefix caching removes only part of that cost. When requests share a system prompt or context, the engine reuses the KV states computed for that prefix, but the request still reaches the model, new tokens are still processed and the full answer is still decoded. Redis describes a prefix-cache hit as a cheaper generation call rather than an avoided one.

LangCache moves the cache outside the model and stores the generated response itself, in what Redis describes as a two-call loop. Before invoking the model, the application sends the prompt to POST /v1/caches/{cacheId}/entries/search; LangCache generates an embedding for the prompt and runs a vector search over stored entries. If a semantically similar entry clears the configured similarity threshold, the cached response is returned and no LLM call occurs. On a miss, the application calls its chosen model as usual and then stores the prompt and new response through POST /v1/caches/{cacheId}/entries for future matches. Embedding generation is handled by the service, using default models or a bring-your-own option. Cache behavior is controlled through similarity thresholds, time-to-live settings and eviction policies, plus adaptive controls that tune precision and recall. Built on Redis's vector database and exposed as a REST API, the service works with any LLM provider and language, with hit rates and savings monitored from the Redis Cloud console.

What a hit removes is the input tokens, the output tokens and the decoding latency of an additional model call. In a demo run comparing both paths on a paraphrased question, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. LangCache returned the earlier response in 0.37 seconds with zero LLM input or output tokens, roughly six times faster in that run. Redis's documentation is more careful about how savings accrue than the headline figures suggest: on a cached response the customer does not pay for output tokens, while input token costs are typically offset by embedding and storage costs. The suggested estimate is monthly output token costs multiplied by cache hit rate. With $200 of monthly LLM spend, 60 percent of it on output tokens and a 50 percent hit rate, that works out to $60 saved per month; Redis also publishes a calculator for annual estimates.

The figures Redis has published have shifted. Its public preview announcement cited up to 15 times faster responses on cache hits and up to 70 percent lower token usage, while the current product page states savings of up to 90 percent. Customer Mangoes.ai reports a 70 percent hit rate on its patient-care voice app, cutting LLM spending by 70 percent with responses four times faster. Redis says the actual result depends on how much safe repetition exists in a given traffic pattern.

Deciding which questions can safely share an answer is a production concern rather than a configuration detail. A threshold set too low returns a refund policy to a customer asking about upgrades; set too high, nearly every paraphrase goes back to the model and the cache stops paying for itself. Redis lists well-tuned thresholds, expiration policies so stale answers age out, data isolation between tenants and monitoring for incorrect matches as requirements. LangCache addresses these with access scopes, custom filtering, TTL and eviction controls and monitoring through Redis Cloud. Data stays on the customer's Redis servers, and Redis states that it does not access that data or use it to train models.