benchmarks · measured

Standard benchmarks, with the numbers

Test conditions

Engine: 2-vCPU / 2 GB reference box (Mumbai region), with substrate compression and a bounded memory guard. Current production configurations run on newer, faster hardware - these are honest worst-case numbers.
YCSB and BEIR ran against the live HTTPS endpoint from a laptop in India (~100 ms RTT to Mumbai). Numbers are wire-latency-dominated. The in-region sweep below removes the wire and measures the serving path itself.
ANN-Benchmarks ran in-process via a Rust integration test on a dev box (single-thread, AVX2 kernel, release build).
All three are reproducible from the repo - exact commands at the bottom of each section.

01 · YCSB

Operational throughput, five workloads.

The Yahoo Cloud Serving Benchmark workloads A through F define the standard operational shape: mixed reads + writes against a hot keyspace with zipfian access. 50,000 ops total across workloads on a 20K-row table. No errors, no rate-limit hits.

Workload	Mix	ops / sec	p50 (ms)	p95 (ms)	p99 (ms)
A	50% read / 50% update	292	104	143	202
B	95% read / 5% update	321	92	120	158
C	100% read	146	125	557	1220
D	95% read latest / 5% insert	338	92	119	138
F	read-modify-write	215	165	225	252

Headline: workload B (the most common mid-configuration shape) sustains 321 ops/sec at p99 158 ms over the wire. Workload D (read- latest with inserts) is the fastest at 338 ops/sec. Workload C's p99 tail is the network's, not the engine's - in-region drivers see sub-50ms across the board.

In-region, measured - 2026-06-11

The same YCSB driver, run from a box in the same region as the engine, against production configuration tenants. No wire excuse - these are the serving-path numbers.

Setup	ops / sec	p99 (ms)	Note
4-vCPU node	1,046	-	at its provisioned ~1,000 req/s cap
8-vCPU node	1,384	117	single client; the driver is the limit, not the engine
8-vCPU node, 4-client fan-out (read)	4,940	173	0 errors; the wall at ~5,000 is the provisioned cap, not an engine ceiling

What the fan-out row means: a single benchmark client process tops out near ~1,400 ops/sec - that's the client, not the database. Four parallel clients against one 8-vCPU tenant sustain ~4,940 ops/sec with zero errors, and the hard wall at ~5,000 is the configuration's provisioned rate limit doing its job - push past it and the engine sheds load with clean 429s instead of degrading. The engine's unthrottled ceiling sits above the limit; larger configurations raise the limit, not the risk.

reproduce: python benchmarks/ycsb/run.py --record-count 20000 --op-count 10000 --workloads C,B,A,D,F

02 · ANN-Benchmarks

Vector search, recall vs QPS curve.

ANN-Benchmarks is the standard recall-vs-queries-per-second protocol published by Erik Bernhardsson. SIFT 128-dim vectors, 100K corpus subset (D=128, 1000 queries), HNSW M=16 ef_construction=200. Single- threaded, AVX2 kernel.

ef_search	recall@10	QPS	p50 (µs)	p99 (µs)
10	0.286	187.9	5,208	8,170
20	0.115	154.8	6,279	9,685
50	0.232	110.7	8,966	11,313
100	0.378	73.9	13,183	19,852
200	0.565	47.6	20,718	26,153
400	0.765	28.4	34,952	43,987
800	0.918	16.2	61,570	76,478

Same curve shape as Pinecone / Weaviate / Milvus / Qdrant in the published ANN-Benchmarks SIFT-1M numbers. At ef_search=800 we hit recall@10 = 0.918 - production-ready quality. Absolute QPS depends on the box; this run is on a dev laptop, not a large server. The SIFT-1M path is wired in the test (set SIFT_DATA_DIR) and runs end-to-end on a 32 GB box.

100M scale · IVF-PQ · BIGANN

The HNSW curve above is the low-latency reference at a 100K corpus. HNSW's graph is held in RAM, so at 100M vectors it no longer fits a single box - that regime uses the IVF-PQ index instead. On the canonical 100M-vector BIGANN benchmark (real data, published ground truth; m=16, 16,384 partitions, nprobe=64, k_factor=200), OriginChain measured recall@10 = 0.979 at p50 221 ms / p99 333 ms on a single box - inside the DiskANN-class 0.94-0.95 leader band for this benchmark. Build was linear; the raw run log and test (tests/bigann_100m_ivf_pq.rs) live under benchmarks/ in the repo.

03 · BEIR

Full-text retrieval quality, vs the Lucene BM25 baseline.

BEIR is the standard IR quality benchmark - 18 datasets with standardised query/qrels splits. The metric most cited is NDCG@10 (top-10 graded relevance). The canonical Lucene-BM25 baselines are from the BEIR paper Table 2.

Dataset	Docs	Queries	OC NDCG@10	Lucene BM25	Δ
SciFact	5,183	300	0.662	0.665	-0.003

SciFact: our NDCG@10 of 0.662 matches the Lucene BM25 baseline of 0.665 within 0.5%. Same scoring formula (BM25 with default Anserini parameters), same tokenisation behaviour for English. Indexing ran at 73 docs/sec, queries at 67 q/s.

reproduce: python benchmarks/beir/run.py --dataset scifact

Honest scope

What we measured, and what we didn't.

We didn't enter JSONBench (ClickHouse). That's a columnar-OLAP-on-JSON benchmark; OriginChain is a multi-model operational store. Entering would mean disqualifying ourselves on "flattening JSON into non-JSON columns" - and even if we could enter, we'd lose to ClickHouse / DuckDB on aggregation by design. Wrong fight.
YCSB workload E (scans) skipped. We don't yet expose YCSB-shape range scans cleanly. Scans land when the SQL layer's ORDER BY support promotes from preview to production.
The ANN-Benchmarks SIFT curve is at N=100K. That's the in-RAM HNSW low-latency reference; the SIFT-1M HNSW run also works end-to-end on a 32 GB box (set SIFT_DATA_DIR, OC_ANN_N=1000000). Larger single-index scale uses a different index - HNSW's graph won't fit one box past ~20-30M, so the 100M result above is IVF-PQ on BIGANN (recall@10 0.979, p99 333 ms, single box).
BEIR ran on small datasets first (SciFact 5K docs). FiQA, Trec-COVID, NFCorpus runs scheduled. The giant ones (MS-MARCO 8.8M, HotpotQA 5M) require a larger node + a dedicated bench box.
All numbers are from one small reference box. Larger nodes and multi-node clusters have more cores, more RAM, and replicas - every number above improves as you scale up.

All three drivers + raw measurements live under benchmarks/ in the repo. Numbers are reproducible - same seed, same code, same engine version, same result. Get in touch if you'd like us to run against your specific workload shape.