How to Reduce Tail Latency on High Throughput RPC
Updated October 22, 2026 · Spectrum Nodes
Short answer: Tail latency is the response time of your slowest requests, usually read at p95 or p99. On a busy RPC fleet it comes from queueing, cold caches and the occasional slow peer. Three levers cut it: reuse connections, coalesce identical in flight reads and route each caller to the node nearest to it. Applied together, they routinely halve p99.
What Is Tail Latency?
Tail latency is how long your slowest requests take. p99 is the time under which 99% of requests finish, so the slowest 1% sit above it. p95 does the same for the slowest 5%.
Average latency hides the requests that hurt. A fast mean can sit on top of the 1 in 100 calls that stalls a trading bot or an indexer.
The gap between median and tail can be wide. In our 14 day test (September 29 to October 14, 2026, test server in Finland, about 50,000 same instant requests per chain and call type), block traces (debug_traceBlockByNumber) measured:
| Chain | Spectrum's Boosted Nodes, median | Spectrum's Boosted Nodes, p95 | Alchemy, median | Alchemy, p95 |
|---|---|---|---|---|
| BSC | 82 ms | 237 ms | 707 ms | 2,182 ms |
| Polygon | 77 ms | 222 ms | 376 ms | 932 ms |
| Base | 126 ms | 399 ms | 409 ms | 2,120 ms |
Where Does p99 Latency Come From on an RPC Fleet?
On a high throughput fleet, the p99 is dominated by three things:
- Queueing. Requests wait behind others when a node is busy.
- Cold caches. A read for data that is not already cached has to go to disk or upstream.
- The occasional slow peer. One lagging node drags every request routed to it.
Which Levers Reduce Tail Latency?
- Connection reuse. TLS and TCP handshakes are pure overhead on a hot path, so keep connections open and reuse them.
- Request coalescing. Identical in flight reads should share one upstream call.
- Locality. Route to the node nearest the caller, not the node nearest to you.
None of these are exotic. Applied together, they routinely halve p99. For more tuning, read RPC optimisation in production and 5 RPC metrics that matter.
How Does Spectrum Nodes Cut Tail Latency?
Spectrum's Boosted Nodes are our caching and routing layer in front of our own nodes, served from regional endpoints. On BSC block traces (debug_traceBlockByNumber) they measured 237 ms at p95 against 2,182 ms on Alchemy, from September 29 to October 14, 2026. The full Polygon results are in our Polygon RPC provider comparison.
FAQ
What is tail latency in RPC? Tail latency is the response time of the slowest requests to an RPC endpoint, usually measured at p95 or p99. It matters more than the average for bots, indexers and automation, because one slow call can hold up a whole job.
What is the difference between p95 and p99 latency? p95 is the time under which 95% of requests finish, and p99 is the time under which 99% finish. p99 captures rarer and slower outliers, so it is usually higher.
What does p95 latency mean for blockchain indexers? An indexer calls the same heavy method on every block, so the slowest 5% adds up across a run. On BSC block traces, Spectrum's Boosted Nodes measured 237 ms at p95 against 2,182 ms on Alchemy (September 29 to October 14, 2026).
What is request coalescing? Request coalescing merges identical reads that are in flight at the same moment into one upstream call. Every caller gets the same answer, and the node does the work once.