Rate limits
Rate limits protect an instance from runaway requests - a buggy loop, a forgotten while True, an autoscaling event. Budgets are attached to each API key, are generous enough for most apps, and scale with your configuration; the goal is to bound damage from accidents, not to gate normal usage.
What's budgeted.
Each API key carries four independent budgets. A request is admitted only if it fits all of them; the first budget you exhaust is the one named in the X-OC-Limit-Hit header on the 429.
| Budget | Counted as | Notes |
|---|---|---|
| requests / sec | API calls per second | The primary budget. One batch insert of 1,000 rows counts as one request - see batching below. |
| bytes / sec | Payload volume per second | Bounds total data movement independently of request count, so a few huge payloads can't starve the instance. |
| NL queries / sec | Natural-language (/ask) calls per second | Budgeted separately because a cold compile is far more expensive than an ordinary query. |
| concurrent queries | Simultaneous in-flight queries | Caps open requests at any instant, independent of per-second rates. |
Budgets are per-key, not per-instance. If you create multiple keys (e.g., one per service), each gets its own budget. The figures for your instance depend on your configuration - and they scale with node count. If you need more headroom, contact support@originchain.ai - production accounts can have budgets raised on request.
Budgets scale with node count.
On a multi-node configuration, every budget multiplies by the node count: a 3-node configuration carries 3x the requests/sec, bytes/sec, NL queries/sec, and concurrent-query budgets of a single node.
You still talk to one endpoint, and that endpoint delivers the full aggregate budget. There is no traffic to split on the client side and no per-node quota to balance - send your traffic to the same URL and the whole contract is available there.
Batch before you scale.
Rate budgets are counted per request, and batch endpoints move many rows per request - which makes batching the single cheapest lever you have. Measured on our standard ingest workload, the batch path delivers roughly 18x the throughput of the same data sent as single-row calls.
If you're hitting 429s on an ingest path, switch to batch inserts before you reach for a bigger configuration. Most "we need more rate limit" conversations end with a batch endpoint.
Response headers.
Every successful response carries your current usage. Use these to back off proactively before you hit the limit.
# Every API response includes these headers:
X-RateLimit-Limit: 1000 # request budget for this key
X-RateLimit-Remaining: 847 # what you have left in the current window
X-RateLimit-Reset: 1714478400 # epoch seconds when the window resets
# On 429 responses you also get:
Retry-After: 12 # seconds to wait before retrying
X-OC-Limit-Hit: bytes_per_sec # WHICH budget you exceeded
When X-RateLimit-Remaining drops near zero, your code can slow itself down voluntarily instead of waiting for the 429 response.
Handling 429.
When you exceed a budget, OriginChain returns 429 rate_limited with a Retry-After header (seconds) and an X-OC-Limit-Hit header naming the budget you exhausted - so you know whether to slow down, shrink payloads, or cap concurrency. The SDKs handle the retry automatically - they read Retry-After, wait, and retry up to 3 times. If you're calling the API directly, do the same:
# The Python SDK handles 429 + Retry-After automatically.
# This is what it does under the hood:
import time
def call_with_retry(fn, max_retries=3):
for attempt in range(max_retries + 1):
try:
return fn()
except OCRateLimitedError as e:
if attempt == max_retries:
raise
time.sleep(e.retry_after or 1.0)// Same logic for raw fetch users:
async function callWithRetry(req: () => Promise<Response>, maxRetries = 3) {
for (let attempt = 0; attempt <= maxRetries; attempt++) {
const res = await req();
if (res.status !== 429) return res;
if (attempt === maxRetries) return res;
const wait = parseInt(res.headers.get("Retry-After") ?? "1") * 1000;
await new Promise(r => setTimeout(r, wait));
}
throw new Error("unreachable");
}// Same logic with stdlib http:
for attempt := 0; attempt <= 3; attempt++ {
resp, err := http.DefaultClient.Do(req)
if err != nil { return err }
if resp.StatusCode != 429 { return nil }
if attempt == 3 { return fmt.Errorf("rate limited") }
wait := 1
if h := resp.Header.Get("Retry-After"); h != "" {
fmt.Sscanf(h, "%d", &wait)
}
time.Sleep(time.Duration(wait) * time.Second)
} - Tight retry loops without backoff. Retrying instantly on 429 just consumes the next minute's quota. Always honor
Retry-After. - Sharing one key across many workers. Budgets are per-key, so a fleet of workers sharing one key shares one concurrent-query budget. Issue one key per worker or per service.
- Bulk inserts in single-row mode. Sending 10,000 single-row inserts will hit the rate limit. Use
_batchinstead - one request, thousands of rows, ~18x the measured throughput. See batching above and Insert → bulk.