Cache

Overview

Aurora ships with two response-cache layers for non-streaming requests on:

• /v1/chat/completions
• /v1/responses
• /v1/embeddings

Exact-match cache returns byte-identical responses with:

X-Cache: HIT (exact)
 

Semantic cache uses embeddings plus vector search so meaning-equivalent prompts can reuse a stored response:

X-Cache: HIT (semantic)
 

Enable the exact cache

Point response caching at Redis:

cache:
  response:
    simple:
      redis:
        url: redis://localhost:6379
        ttl: 3600
 

You can also configure it with environment variables:

• RESPONSE_CACHE_SIMPLE_ENABLED=true
• REDIS_URL
• REDIS_KEY_RESPONSES
• REDIS_TTL_RESPONSES

RESPONSE_CACHE_SIMPLE_ENABLED=true is the opt-in switch for env-only deployments when config.yaml does not contain a cache.response.simple block.

Enable semantic caching

Add a semantic block with an embedder provider and a vector store:

cache:
  response:
    semantic:
      enabled: true
      embedder:
        provider: openai
      vector_store:
        type: qdrant
        qdrant:
          url: http://localhost:6333
          collection: aurora_semantic
 

Supported vector stores:

• qdrant
• pgvector
• pinecone
• weaviate

Env-only semantic cache deployments use the SEMANTIC_CACHE_* variables:

SEMANTIC_CACHE_ENABLED=true
SEMANTIC_CACHE_EMBEDDER_PROVIDER=openai
SEMANTIC_CACHE_EMBEDDER_MODEL=text-embedding-3-small
SEMANTIC_CACHE_VECTOR_STORE_TYPE=qdrant
SEMANTIC_CACHE_QDRANT_URL=http://localhost:6333
SEMANTIC_CACHE_QDRANT_COLLECTION=aurora_semantic
 

All semantic cache environment variables:

Both cache layers run after workflow and guardrail patching, so they operate on the final request sent upstream. Use Cache-Control: no-cache or Cache-Control: no-store to bypass caching per request.

For the full semantic-cache design and storage options, see the gateway source configuration examples and .env.template.

Per-request cache overrides

These request headers let you pin cache behavior for a single call without changing global config:

Pin the cache to one layer when you are tuning and want to isolate the variable. Bypass per request when you are debugging a one-off miss.

What the exact cache keys on

The exact cache hashes:

• the request path
• the resolved workflow context used for execution specifically execution mode, provider type, and resolved model
• the final request body

This means guardrails and workflows affect cache keys when they change the resolved workflow or the final body sent through execution.

user_path behavior

For the exact cache, user_path is not added to the cache key by itself.

That is intentional. If two requests end up with the same path, resolved workflow, and final request body, they can share the same exact-cache entry even when they originate from different user_path values.

Common patterns:

• disable cache in a scoped workflow
• use different scoped workflows for different user_path values
• include scope-specific context so the final request body differs

Cache analytics

When response caching and usage tracking are enabled, the admin API exposes a cached-only overview at:

/admin/api/v1/cache/overview
 

Cached usage entries are also visible in the regular usage log and summary endpoints.

Manual and API usage

Cache backends are boot configuration. Enable exact or semantic cache through YAML or environment variables before starting Aurora. Per-scope cache behavior is controlled by workflows, so operators can manually disable cache for a team, provider, or model in the dashboard at Workflows.

Server-side automation should use workflows for policy changes and cache debug for explaining a specific request:

For endpoint reference see the Admin API section.

POST /admin/api/v1/cache/debug

Use this endpoint to ask Aurora "would this request have hit the cache, and why or why not?" without actually sending the upstream call.

Request:

curl -X POST http://your-aurora-host/admin/api/v1/cache/debug \
  -H "Authorization: Bearer $AURORA_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "method": "POST",
    "path": "/v1/chat/completions",
    "headers": {"Authorization": "Bearer sk-aurora-..."},
    "body": {
      "model": "openai-primary/gpt-4o-mini",
      "messages": [{"role": "user", "content": "Hello"}]
    }
  }'

method defaults to POST; path defaults to /v1/chat/completions. The endpoint only works when at least one cache layer is enabled; otherwise it returns feature_unavailable.

Response (real shape from internal/response_cache/middleware.go:49):

{
  "path": "/v1/chat/completions",
  "cache_type": "exact",
  "exact_cache_key": "aurora:response:sha256:...",
  "semantic_params_hash": "",
  "semantic_cache_key": "",
  "semantic_threshold": 0.0,
  "prompt_similarity_threshold": 0.0,
  "exact_ttl_seconds": 3600,
  "semantic_ttl_seconds": 0,
  "streaming": false,
  "cacheable": true,
  "miss_reason": "",
  "guardrails_hash": "sha256:...",
  "embedder_identity": "openai/text-embedding-3-small",
  "effective_content_type": "application/json"
}

Field meanings:

• cache_type � the layer that would have been consulted (exact, semantic, or empty if disabled for the matched workflow)
• exact_cache_key / semantic_cache_key � the hashes Aurora would have used
• semantic_threshold / prompt_similarity_threshold � the configured similarity thresholds; useful for tuning
• exact_ttl_seconds / semantic_ttl_seconds � the configured TTLs
• cacheable: false � the matched workflow or a request header is blocking caching
• miss_reason � when cacheable: true and a real call would miss, the reason (e.g. streaming_mode_keyed_separately, fallback_served)
• guardrails_hash � the hash of the resolved guardrail chain; identical hashes mean an identical upstream-facing body
• embedder_identity � which embedder the semantic cache would use
• effective_content_type � the content type Aurora would have stored