Failover

Overview

Aurora exposes failover through the fallback config block.

When a request fails, Aurora can retry it against alternate models. For predictable behavior, use manual mode.

Failover is an OSS-capable gateway feature. Enterprise advanced routing can add additional adaptive routing controls, but manual fallback chains do not require Enterprise.

Manual Mode

Manual mode is the recommended mode today.

fallback:
  default_mode: "manual"
  manual_rules_path: "configs/fallback.json"
 

configs/fallback.json is a JSON object where each model entry contains an ordered candidate list (array); top-level keys are not ordered:

{
  "<primary-model-id>": [
    "<provider-name>/<first-fallback-model-id>",
    "<provider-name>/<second-fallback-model-id>"
  ]
}
 

The order-sensitive part is the array under each model entry.

Aurora tries the listed candidates in order and stops on the first success. Use bare model names or provider-qualified selectors from GET /admin/api/v1/models.

If needed, you can override the mode per model with fallback.overrides.

Auto Mode

fallback:
  default_mode: "auto"
  manual_rules_path: "configs/fallback.json"
 

Auto mode keeps any manual candidates first, then appends up to five extra candidates from the current model registry. It prefers models with the same request category, similar rankings, overlapping capabilities, and the same family when possible.

When It Runs

Failover is attempted only after the primary request returns:

• 5xx
• 429
• model unavailable, unsupported, or not found style errors

It currently applies to translated /v1/chat/completions and /v1/responses requests, not /v1/embeddings.

Manual and API usage

Manual fallback rules are boot configuration. Edit fallback.manual_rules_path and restart or refresh runtime state if your deployment supports runtime refresh.

Server automation can update the JSON rules file in your deployment system and then call:

curl -X POST http://your-aurora-host/admin/api/v1/runtime/refresh \
  -H "Authorization: Bearer $AURORA_MASTER_KEY"
 

Use workflows to turn fallback behavior on or off for a provider, model, or user_path scope. See Workflows.

What to click in the dashboard

• Providers page shows per-provider health, error counts, and last error. A provider in red is a candidate for a manual rule that maps its models to a backup.
• Pools page shows pool health, member distribution, and per-member error counts. A pool that is round-robining unevenly has a health problem on one member.
• Workflows page is where you turn fallback on or off for a specific (provider, model, user_path) scope. The default workflow enables fallback globally; a scoped workflow can disable it for a single team.
• Audit Logs page filtered by status_code >= 500 shows the requests that triggered failover. The model field on the response is the actual upstream that answered (could be a fallback target), not the alias the client sent.
• Console page logs each failover transition in real time.