Advanced Routing

--- title: "Advanced Routing" description: "Adaptive provider pool routing with latency-aware, error-aware member selection for Enterprise deployments." icon: "route" ---

Overview

Enterprise advanced routing extends the OSS provider pool system with adaptive strategy selection. While OSS pools support round-robin, least-loaded, weighted, and random strategies, Enterprise adds the adaptive strategy — a self-tuning routing approach that scores each pool member based on real-time error rates, latency, utilization, and configured weights.

Adaptive Routing Strategy

The adaptive strategy selects pool members probabilistically using a composite score:

score = errorPenalty × 0.50
      + latencyPenalty × 0.20
      + utilizationPenalty × 0.05
      + weightFactor

Scoring Factors

EWMA Latency Tracking

Latency is smoothed using an 80/20 EWMA:

latencyEWMA = 0.8 × previousEWMA + 0.2 × latestLatency

This provides fast adaptation to changing conditions while filtering out noise.

Configuration

Prerequisites

The advancedRouting Enterprise capability must be enabled. This is included automatically when edition.name: enterprise.

Adaptive Pool Configuration

pools:
  my-pool:
    strategy: adaptive
    health_aware: true
    members:
      - provider_name_1
      - provider_name_2
      - provider_name_3

All Pool Strategies

Weighted Members

pools:
  my-pool:
    strategy: weighted
    members:
      - name: provider_primary
        weight: 3
      - name: provider_secondary
        weight: 1

The primary provider gets ~75% of traffic; secondary gets ~25%.

Pool Call Styles

Model Prefix

{
  "model": "my-pool/gpt-4o",
  "messages": [{"role": "user", "content": "Hello"}]
}

Provider Field

{
  "model": "gpt-4o",
  "provider": "my-pool",
  "messages": [{"role": "user", "content": "Hello"}]
}

Health and Retry

Pool dispatch checks member health status, latency, and active request counters before routing. On failure, the pool retries against the next best member if available.

The health_aware: true setting excludes unhealthy members from routing until they recover.

Admin API

For endpoint reference see the Admin API section.

curl -X PUT http://your-aurora-host/admin/api/v1/pools/my-pool \
  -H "Authorization: Bearer $AURORA_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "strategy": "adaptive",
    "health_aware": true
  }'

Dashboard

Pool management and health inspection are available in the admin dashboard at Pools. The dashboard shows per-member health, latency, and routing distribution.

Related

• Provider Pools — OSS pool basics and setup
• Failover — Cross-model fallback configuration