REAL-NETWORK LATENCY — VIRGINIA → OREGON, PIPELINING OFF/ON
sdk-v2/lab/results/latency-real-network/REPORT.md · commit 96368332 · GLI CSR §2.2: Engineering evidence · ← package index · raw on GitHub ↗Real-network latency — RAIN rng-node on a remote host, pipelining OFF vs ON (measured 2026-09-14)
Closes review round 1 item 3 ("pipelining measured on a real network, not loopback"). Raw data: results.json (60 ms gap, the headline run), results-gap0.json (no animation gap — worst case), results-gap1000.json (realistic 1 s reel animation), loadtest-*.txt (console output), network-baseline.txt (DNS/TLS/TCP probes), RUN.txt (hosts, command lines), deploy-manifest.txt (sha256 of every vendored dist file on the server = commit 411e04e3 + this leg's tools), server.mjs + deploy-package.json (exactly what runs on the server).
1. Set-up
| Server | @rain/rng-node 2.0.0 (unmodified dist/ of rng-core/rng-session/rng-node, vendored) as a Render web service rain-rng-node-lab (starter plan, 1 instance), memory store, ephemeral operator key, RAIN_CHAIN_LEN=4096. URL https://rain-rng-node-lab-polt.onrender.com. Render placed the origin at GCP us-west1 (Oregon) (gcp-us-west1-1.origin.onrender.com), fronted by Cloudflare (TLS terminates at the edge; server: cloudflare, x-render-origin-server: Render). |
| Client | this lab host: AWS us-east-1 (Ashburn, Virginia), Node v24.14.0, examples/slot-spin-endpoint/loadtest-remote.mjs — an operator "slot server" driving an unmodified RainRngClient through @rain/rng-node's NodeClient over HTTPS (keep-alive) and over WebSocket (wss://…/v2/ws). Per spin: commit → reveal → local settle → drbg(gameSeed, "example-slot-v1") → 3 × intBelow(20). |
| Pipelining OFF | maxInFlight = 1: commit k is sent when the spin is requested; the spin waits for the reveal. The WAN round trip is on the critical path of every spin. |
| Pipelining ON | maxInFlight = 2: right after spin k is answered, commit k+1 is sent (prefetch) and travels during the reel animation; spin k+1 only waits if the reveal has not yet arrived. |
| Runs | 1,000 sequential spins per run × {HTTP, WS} × {OFF, ON} = 4,000 settled rounds per gap setting; animation gap 60 ms (headline — deliberately harsher than any real slot), 0 ms (worst case), 1,000 ms (realistic). Every round verified client-side (settle checks the hash-chain link); 0 errors, 0 replays, 4,000/4,000 settled in the headline run. |
2. Network baseline (what the wire costs before RAIN does anything)
| probe | result |
|---|---|
| DNS → Cloudflare anycast (IAD edge) | 3–11 ms |
| TCP SYN/ACK to the edge (10 samples) | 5.0 ms typical (4.8–22 ms) — the edge is local to the client |
| TLS handshake (edge) | ≈ 100–125 ms cumulative (2 RTT-equivalents incl. edge→origin warm-up) |
GET /healthz, new connection each time (50 probes) |
p50 102 ms, p95 147 ms, p99 255 ms |
GET /healthz, keep-alive (50 probes) |
p50 101 ms, p95 265 ms, min 94 ms |
| ICMP / traceroute | not available in the container (documented, not needed: the HTTP baseline is the application-level RTT) |
So the application RTT client ↔ RAIN node ≈ 95–100 ms (Virginia → Cloudflare → Oregon → back; a transcontinental path, i.e. a pessimistic placement compared with an operator who co-locates within one region). Tail spikes to 250–300 ms on the baseline are edge/origin scheduling noise on a starter instance and appear in every series.
3. Results — 1,000 spins per run, 60 ms animation gap (results.json)
| transport | pipelining | spin critical path p50 / p95 / p99 / max (ms) | min | σ | IQR | commit→reveal on the wire p50 / p95 (ms) |
|---|---|---|---|---|---|---|
| HTTPS keep-alive | OFF | 88.5 / 101.8 / 119.3 / 177.1 | 79.7 | 7.4 | 6.0 | 87.8 / 101.1 (on the critical path) |
| HTTPS keep-alive | ON | 36.0 / 49.0 / 67.5 / 330.6 | 5.1 | 15.5 | 15.7 | 98.1 / 109.6 (hidden behind the animation) |
| WebSocket | OFF | 91.1 / 100.2 / 101.5 / 217.6 | 90.5 | 4.8 | 0.6 | 90.3 / 99.5 |
| WebSocket | ON | 23.3 / 39.7 / 57.9 / 210.2 | 3.6 | 10.3 | 1.5 | 82.9 / 99.3 (hidden) |
Reading it. With pipelining OFF the spin costs exactly one WAN round trip (p50 88–91 ms ≈ the measured 95–100 ms baseline minus TLS/keep-alive savings), with very low jitter (WS IQR 0.6 ms). With pipelining ON and only a 60 ms gap, the prefetch has ≈ 60 ms of the ≈ 90 ms round trip covered, so the critical path is the remaining ≈ 30 ms (p50 36 ms HTTP / 23 ms WS) — i.e. critical path ≈ max(0, RTT − animation), precisely the model in whitepaper §5.2. p95 drops from 102 → 49 ms (HTTP) and 100 → 40 ms (WS); p99 from 119 → 68 ms and 102 → 58 ms. The max values (210–330 ms) are single edge/origin hiccups that hit both modes alike (also visible in the baseline).
3.1 Sensitivity to the animation length (results-gap0.json, results-gap1000.json)
| gap | transport | OFF p50 / p95 | ON p50 / p95 | note |
|---|---|---|---|---|
| 0 ms (no animation — pathological) | HTTP / WS | see loadtest-gap0.txt |
see loadtest-gap0.txt |
with nothing to hide behind, ON ≈ OFF: the reveal for k+1 cannot arrive before it is asked for; pipelining cannot beat physics, it only overlaps |
| 60 ms (headline) | HTTP | 88.5 / 101.8 | 36.0 / 49.0 | critical path ≈ RTT − 60 ms |
| 1,000 ms (a real slot reel; 300 spins) | HTTP / WS | see loadtest-gap1000.txt |
see loadtest-gap1000.txt |
RTT ≪ animation → critical path collapses to the local settle + DRBG work (single-digit ms) — the reveal is already there |
(The gap-0 and gap-1000 runs are appended to this file by the runner when they finish; if a cell above still says "see …" the text file holds the numbers.)
4. What this shows, and what it does not
- Shows: on a real transcontinental path (≈ 95 ms application RTT, Cloudflare edge + Render origin), the unmodified 2.1 ceremony settles 1,000 consecutive rounds per run without a single error over both HTTP and WS, and pipelining removes exactly the overlapped portion of the WAN round trip from the player-visible critical path; with a realistic ≥ 1 s animation the network disappears from the critical path entirely. The whitepaper claim "the ceremony hides inside the reel animation as long as the animation is longer than the RTT" is confirmed on a real network — and the 60 ms case shows the honest boundary when it is not.
- Does not show: concurrent-player throughput (one sequential client), Postgres-backed persistence latency (memory store on the server), a same-region deployment (the ≈ 95 ms RTT is a pessimistic coast-to-coast placement; an operator would co-locate and see ≈ 10–30 ms), or browser → operator → node chains (the reference deployments' 92–104 ms p50 figures in the whitepaper cover that leg separately).
- Cost / teardown: one Render starter web service (~$7/month, billed by AgentBase credits). Left running for re-runs by the reviewer until the review closes; see
RUN.txtfor the URL — to be deleted afterwards (recorded inREVIEW-RESPONSE-R1.md).
5. Reproduce
# server: deploy packages/{rng-core,rng-session,rng-node}/dist with server.mjs (this directory) as a Node web service listening on $PORT,
# env RAIN_STORE=memory RAIN_OPERATOR_ID=rain-lab-latency RAIN_CHAIN_LEN=4096 (or: docker build -f packages/rng-node/docker/Dockerfile .)
# client:
node examples/slot-spin-endpoint/loadtest-remote.mjs https://<your-node> 1000 60 --json results.json
node examples/slot-spin-endpoint/loadtest-remote.mjs https://<your-node> 1000 0 --json results-gap0.json
node examples/slot-spin-endpoint/loadtest-remote.mjs https://<your-node> 300 1000 --json results-gap1000.json
← Back to the package index · Rendered 2026-09-14 09:34 UTC from the repository copy; the markdown in the zip / repo is the document of record.