PROCESS/HOSTING-HA-DR
sdk-v2/lab/process/HOSTING-HA-DR.md · commit 96368332 · GLI CSR §2.2: §2.11 hosting / HA / DR · ← package index · raw on GitHub ↗HOSTING, HIGH AVAILABILITY, DISASTER RECOVERY — reference deployment of @rain/rng-node (GLI CSR §2.11; GLI-19 App. B; RTS 8.13/8.15/8.17; MGA Art. 17(4))
This is a per-operator document; what follows is the reference architecture shipped in the repository and what has and has not been executed.
1. Components
- Workers:
rain-rng-node(Node.js 24,packages/rng-node/docker/Dockerfile), stateless; N replicas behind nginx (docker/nginx.conf,docker-compose.ha.yml). Endpoints:/v2/*(RNG),/verify(public proof),/healthz,/selftest,/metrics. - Store (single point of truth): Postgres with
synchronous_commit=on(schemaPG_SCHEMA,CREATE … IF NOT EXISTS) — or the file WAL for a single node. Persist-before-reveal relies on the store's durability (fsync). - Optional anchoring:
RngAnchor.solon an EVM chain; hot wallet with gas. - Time: NTP on hosts;
/healthzchecks skew againstRAIN_TIME_URL(clock does not affect randomness; it affects log ordering and dispute windows).
2. Network (template — fill per deployment)
Internet → WAF/LB (TLS termination) → nginx → workers (private subnet) → Postgres (private subnet, no public route). Only the LB is public. Firewall: allow 443 in; workers egress only to Postgres, NTP, anchoring RPC, attribution URL. Diagram with IPs/VLANs to be attached by the operator.
3. HA behaviour
Any worker serves any session (state is in the store); rolling restarts are safe; a worker crash between commit-persist and reveal leaves a "persisted commit without reveal" that any worker serves idempotently (GET /v2/reveal/:sid/:k). Executed evidence: in-process two-worker test sharing one store with a client alternating workers per round and cross-worker /verify (test/e2e.node.test.mjs), crash injection at every persist boundary (test/crash.test.mjs). Not executed in this environment: scripts/ha-test.sh under real Docker (Docker unavailable) and the Postgres store against a live server — the operator must run both before go-live and attach the output.
4. Backup / restore
- Postgres: streaming replica + PITR (WAL archiving); RPO ≈ 0 with synchronous replica; RTO minutes.
- File WAL: append-only
wal.jsonl;rsync --append/object-storage sync on schedule and after every deploy. - Never restore a store older than the live one (would allow a second reveal path to be requested → clients get
order/bad_prev; treat as S1 incident). Two workers must never point at diverged stores.
5. Regulator mirror (MGA Art. 17(4) "live or real-time mirror server for essential regulatory data")
Replicate the Postgres sessions/commits/reveals tables to a regulator-readable replica, or publish anchors on-chain — both give a third party the commitments and reveals needed to recompute every round.
6. Logging (NJ 13:69O-1.8)
Every commit and reveal is a WAL/DB record with sessionId, k, decision, pRev, hRev, timestamps; node logs are JSON lines. Retention per jurisdiction (≥ 5 years typical). /verify/:sid/:k reproduces the full audit view for one round.
7. Capacity
Measured RNG work on the /spin path p95 0.5 ms (docs/RNG.md §2.6); throughput is store-fsync bound. Chain generation per session ~ms for 4 096–65 536 elements.
← Back to the package index · Rendered 2026-09-14 09:34 UTC from the repository copy; the markdown in the zip / repo is the document of record.