examples · fts · 5 / 6

5. BM25 ranked retrieval

what this does

Returns the top k documents ranked by BM25 relevance to the query. Plain BM25 returns a bare JSON array - each entry is { doc_id, score }, sorted by descending score. Higher score = more relevant. (Boolean and phrase modes return arrays of plain doc_id strings; BM25 adds the score.)

when to use it

Search bars - users expect the best match at the top.
Recommendations: rank a candidate pool by textual similarity.
Any time you want "best k" rather than "all that match".

the request

GET /v1/tenants/:t/fts/:schema/:field?q=...&mode=bm25&k=10

curl -G "https://$OC_HOST/v1/tenants/$OC_TENANT/fts/shop.products/description" \
  -H "Authorization: Bearer $OC_TOKEN" \
  --data-urlencode "q=wireless headphones" \
  --data-urlencode "mode=bm25" \
  --data-urlencode "k=10"

# Plain BM25 returns a bare JSON array of { doc_id, score } objects
hits = db.fts.search(
    "shop.products",
    "description",
    q="wireless headphones",
    mode="bm25",
    k=10,
)
for hit in hits:
    print(hit["doc_id"], hit["score"])

// Plain BM25 returns a bare array of { doc_id, score }
const hits = await db.ftsSearch("shop.products", "description", {
  q: "wireless headphones",
  mode: "bm25",
  k: 10,
});
for (const hit of hits) {
  console.log(hit.doc_id, hit.score);
}

// Plain BM25 returns []{ DocID, Score }
hits, _ := db.FTSSearch(ctx, "shop.products", "description", originchain.FTSSearchRequest{
    Q:    "wireless headphones",
    Mode: "bm25",
    K:    10,
})
for _, hit := range hits {
    fmt.Println(hit.DocID, hit.Score)
}

what you get back

[
  { "doc_id": "p001", "score": 9.42 },
  { "doc_id": "p027", "score": 7.18 },
  { "doc_id": "p014", "score": 4.55 }
]

A bare array of { doc_id, score } objects - no { "mode": ..., "hits": [...] } wrapper.

the enriched shape (highlight / facets only)

The plain bare-array shape only changes when you add highlight=true or facets=. Then - and only then - BM25 returns an object with a hits array (each hit carrying optional highlights) and a facets block. Highlights require the doc text to have been stored via POST /fts/:t/:f/doc first.

{
  "hits": [
    {
      "doc_id": "p001",
      "score": 9.42,
      "highlights": {
        "description": ["Over-ear <em>wireless</em> <em>headphones</em> with…"]
      }
    },
    { "doc_id": "p027", "score": 7.18, "highlights": {} }
  ],
  "facets": {
    "category": [{ "value": "electronics", "count": 2 }]
  }
}

k, fuzzy, highlight, facets, and explain are BM25-only - they are silently ignored in boolean and phrase modes.

how it works

The query is tokenised, then for each token the engine fetches the posting list along with term frequencies and document lengths.
Each candidate document gets a BM25 score using the Lucene defaults: k1 = 1.2, b = 0.75. Score grows with how often the rare terms appear and shrinks if the document is much longer than average.
A top-k heap keeps the highest scorers; everything else is discarded.

Optional query params: fuzzy=1 for single-character typo tolerance, highlight=true to return matched-snippet text, facets=col,col for grouped counts alongside hits.

common mistakes

Comparing scores across queries. A BM25 score of 9.4 means nothing on its own and can't be compared to the 9.4 from a different query. Use scores to rank within one result set only.
Forgetting k. If you omit it, you get a default cap. Set k to what you actually need; ranking the entire corpus is wasted work.
Reaching for BM25 when boolean would do. If you only need "does it match", boolean is cheaper and the answer is the same set.