Gateway-level data scrubbing is a critical security boundary for any AI infrastructure. When user prompts flow through a routing layer, Personally Identifiable Information (PII) must be intercepted before it reaches upstream LLM providers. The challenge is doing this at line speed without adding perceptible latency to the request path.
At Aurora, we treat PII masking as a first-class pipeline stage rather than a bolt-on filter. This architectural decision allows us to maintain sub-millisecond overhead even under full production throughput.
The Pipeline Architecture
The masking pipeline is structured as a series of composable stages, each responsible for a specific PII category. This design enables targeted performance tuning and parallel execution where possible.
Request Inbound │ ▼ ┌─────────────────────────────────┐ │ Stage 1: Token Boundary Scan │ ~0.1ms ├─────────────────────────────────┤ │ Stage 2: Regex Pattern Matcher │ ~0.3ms ├─────────────────────────────────┤ │ Stage 3: ML Classifier (opt.) │ ~0.8ms ├─────────────────────────────────┤ │ Stage 4: Contextual Redaction │ ~0.2ms └─────────────────────────────────┘ │ ▼ Upstream Provider
Stages 1 and 2 run unconditionally on every request and account for 95% of detections. Stage 3 is invoked only when pattern confidence falls below a configurable threshold, making the pipeline fast by default.
Token Boundary Scanning
The fastest detection path operates on token boundaries rather than raw character streams. By leveraging the same tokenizer used for LLM context windows, we can identify potential PII segments at near-zero cost since the tokenization step is already required for the upstream request.
The scanner maintains a lightweight trie of known PII patterns mapped to token sequences. This structure enables O(n) scanning with memory overhead under 2MB for all common PII categories.
Regex Performance Benchmarks
Our regex engine is compiled at startup into a deterministic finite automaton (DFA). This eliminates backtracking and guarantees linear-time matching regardless of input complexity.
Pattern │ Matches/sec │ p50 Latency ─────────────────────────┼─────────────┼───────────── Email │ 2,400,000 │ 0.04ms Phone (US intl.) │ 1,800,000 │ 0.06ms SSN │ 3,100,000 │ 0.03ms Credit Card │ 1,200,000 │ 0.08ms IP Address │ 2,700,000 │ 0.04ms API Key / Token │ 1,500,000 │ 0.07ms
These benchmarks were measured on a single c7g.xlarge instance with concurrent request simulation at 5,000 RPS. No single pattern exceeds 0.1ms p50 latency.
Contextual Redaction
Simple regex-based masking produces false positives in domain-specific contexts. For example, an alphanumeric string that matches a credit card pattern may be a harmless internal identifier. Our contextual redaction stage maintains a allowlist cache keyed by tenant and workspace, reducing false-positive redactions by over 40% in production deployments.
The cache is implemented as a concurrent Bloom filter with a small LRU for recent matches, keeping memory under 1MB per tenant while achieving hit rates above 99.8%.
Zero-Trust Guarantees
Masking operates on a zero-trust model: data is scrubbed at the inbound edge before any tenant routing or provider selection occurs. This ensures that even if a downstream component is compromised, PII never reaches it in plaintext.
The masking layer is also applied to streaming responses, catching PII that LLMs may occasionally leak in generated output. Response masking runs on partial token chunks to minimize buffering latency.
Production Configuration
pipeline:
masking:
stages:
- token_scan
- regex_match
- contextual_redact
patterns:
email: true
phone: true
ssn: true
credit_card: false
ip_address: false
api_key: true
ml_classifier:
enabled: true
confidence_threshold: 0.85
model: "pii-detector-v3"
allowlist:
cache_ttl: 300s
max_entries: 100000This configuration runs in Aurora deployments handling over 10 million requests daily with p99 masking overhead of 1.2ms — well within the margin for gateway-level processing.
Key Takeaways
- Token-boundary scanning reuses existing tokenization for near-zero-cost PII detection.
- DFA-compiled regex engines guarantee linear-time matching with sub-0.1ms per pattern.
- Contextual allowlisting reduces false positives by 40% without compromising security.
- Zero-trust edge masking protects against downstream compromise.
- Streaming response masking catches LLM-generated PII leaks in real time.