BM25 Scoring

​

Basic Usage

BM25 scores measure how relevant a score is for a given query. Higher scores indicate higher relevance.

The score_bm25 function takes a query and returns a table with two columns: the key_field of each result and its BM25 score. score_bm25 accepts the same parameters as search.

SELECT * FROM <index_name>.score_bm25('<query>');

SELECT * FROM search_idx.score_bm25(
	'description:keyboard',
	limit_rows => 10
);

​

Joining BM25 Scores

Because score_bm25 generates a new score_bm25 column, the function does not return all the columns of the original table. To map the columns of the original table to the output of score_bm25, perform a JOIN on the key_field column.

-- Here, the key_field is "id"
WITH scores AS (
    SELECT * FROM search_idx.score_bm25(
      'description:keyboard',
      limit_rows => 10
    )
)
SELECT scores.id, description, score_bm25
FROM scores
LEFT JOIN mock_items ON scores.id = mock_items.id;

 id |       description        | score_bm25
----+--------------------------+------------
  2 | Plastic Keyboard         |  3.2668595
  1 | Ergonomic metal keyboard |  2.8213787
(2 rows)

​

Score Refresh

Unlike search, score_bm25 bypasses the Postgres index access method and queries the index directly to obtain the BM25 score. As a result, previously-deleted rows may influence the BM25 score.

Running VACUUM on the underlying table will remove all dead rows from the index and ensures that only rows visible to the current transaction are factored into the BM25 score.

VACUUM <table_name>;