Benchmarks

Benchmark snapshot

This page presents real benchmark results comparing Aurora against six other AI gateways on the same machine, under identical rate-limited load.

At a glance

In this benchmark run, Aurora led on every operational signal:

• Throughput: 4,986 req/s at a 5,000 req/s target — highest among all gateways tested
• Zero errors: 100% success rate for the full 60-second run
• Latency: 17.7 ms P50, 47.2 ms P99 — 2–4× lower than the next-fastest gateway
• Memory efficiency: ~72 allocs/op, ~5.9 KB/op — comparable to compiled C-gateways (Kong, APISIX)

Test shape

• OpenAI-compatible /v1/chat/completions endpoint
• Rate-limited load at 5,000 req/s for 60 seconds
• All gateways forward to the same mock OpenAI-compatible backend (local Go server with pre-encoded responses)
• Load generator: aurora-bench-cli with QPC-based timing on Windows
• Each gateway tested independently (sequential runs, not concurrent)

Machine

Benchmarked on a consumer gaming laptop with no server-class optimizations (no NUMA pinning, no CPU governor tuning). Server hardware with more cores and memory will improve all absolute numbers — the relative ordering between gateways should remain similar.

Seven-gateway comparison

All gateways configured for minimal overhead per their official documentation (usage tracking, logging, and non-essential features disabled). Rate target: 5,000 req/s.

Key readouts

• Aurora matched the rate target — 4,986 req/s is 99.7% of the 5,000 target. Only Kong and APISIX came close.
• Aurora is the only gateway with zero errors — all competitor runs had timed-out requests under load.
• Aurora P99 latency (47 ms) is ~2× better than Kong (98 ms) and APISIX (109 ms), and 6× better than Bifrost (286 ms).
• LiteLLM (Python) collapses under load — achieves only 117 req/s (2.3% of target) with 43% error rate and 24-second P99 latency.
• Portkey (Node.js) struggles at 978 req/s with 481 ms P99 — garbage collection overhead at high throughput.

Aurora internal comparison

How Aurora's different configuration modes compare at the same 5,000 req/s target:

Usage tracking and request logging cost ~78% throughput at 5K target when enabled — they are now disabled by default.

Why this data matters

Aurora achieves this on a 4-core, 8 GB laptop — not a 32-core server. The Go runtime's lightweight goroutines and h2c multiplexing keep overhead low even under sustained 5K req/s load.

If you are evaluating gateways, reproduce the test against your own traffic and infrastructure. All benchmark scripts ship with the repository.

Current benchmarks

For Aurora gateway-overhead benchmarking, use the gateway-stack benchmarks in internal/server/gateway_stack_bench_test.go. These measure middleware, routing, chat completions, streaming conversion, usage hooks, and passthrough without external provider noise:

go test ./internal/server -run "^$" -bench BenchmarkGatewayStack -benchtime 10s -benchmem

Historical Aurora vs LiteLLM benchmark

Earlier benchmarks (March 2026) compared Aurora against LiteLLM only at low concurrency (1, 4, 8). Those results are superseded by the seven-gateway comparison above but remain in the repository for reproducibility.

Scripts

The benchmark suite lives in scripts/benchmarks/:

Quick start

.\scripts\run-benchmark.ps1 -Mode smoke -NoWindow

This runs Aurora alone at 500 req/s for 30s. Results land in bench-results/.

Full seven-gateway comparison

.\scripts\run-benchmark.ps1 -Mode publish -NoWindow

This runs all seven gateways in sequence, each at 5,000 req/s for 60s. Full comparison JSON is generated with delta tables.