Provider Pools

--- title: "Provider Pools" description: "Group compatible providers behind one pool for load balancing, failover, and pooled embeddings or generation calls." icon: "network" ---

Overview

Provider pools let Aurora dispatch a request across multiple compatible provider instances. Use pools when you want one logical target backed by several upstreams for higher availability, traffic spreading, or regional routing.

Manage pools in the admin dashboard at Pools, or automate pool reads and strategy updates through the admin API.

Call styles

Aurora supports two pool call styles.

Pool name as model prefix

Put the pool name before the model ID:

{
  "model": "jina-pool/jina-embeddings-v3",
  "input": ["Hello from Aurora."]
}

Aurora recognizes jina-pool as a pool and dispatches to an eligible member.

Pool name in provider field

Keep the model ID unchanged and send the pool in provider:

{
  "model": "jina-embeddings-v3",
  "provider": "jina-pool",
  "input": ["Hello from Aurora."]
}

This is useful when an SDK preserves the model string exactly.

Health and retry behavior

Pool dispatch uses member health, latency, and active request counters. On transient upstream failures such as 5xx, 429, or network errors, Aurora can retry on another eligible member before returning an error to the client.

Pool member counters update through admin APIs and the dashboard so operators can see traffic distribution and error concentration.

Manual setup

Use Providers to add provider instances first. Each provider instance has a configured provider name, provider type, optional base URL, optional API version, API key, and optional model list. Then use Pools to inspect pool membership and change supported pool strategy settings.

Provider instances can also be defined statically in YAML:

providers:
  openai-primary:
    type: openai
    api_key: "${OPENAI_API_KEY}"
  local-vllm:
    type: vllm
    base_url: "http://localhost:8000/v1"

Server API automation

For endpoint reference see the Admin API section.

Example pool strategy update:

curl -X PUT http://your-aurora-host/admin/api/v1/pools/default \
  -H "Authorization: Bearer $AURORA_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "strategy": "weighted",
    "weights": {
      "openai-primary": 80,
      "local-vllm": 20
    }
  }'

Only use provider and pool names that exist in your Aurora instance. Read GET /admin/api/v1/providers/status and GET /admin/api/v1/pools before writing automation that updates weights.

Related features

Use provider pools together with:

• Failover for explicit fallback chains
• Resilience for retry and circuit-breaker tuning
• Routing Scopes when pool behavior should vary by team or tenant