INSERT
/UPDATE
/COPY
/VACUUM
, the BM25 index runs a compaction process that looks for opportunities to merge segments
together. The goal is to consolidate smaller segments into larger ones, reducing the total number of segments and improving query performance.
Segments become candidates for merging if their combined size meets or exceeds one of several configurable layer thresholds. These thresholds define target
segment sizes — such as 10KB
, 100KB
, 1MB
, etc. For each layer, the compactor checks if there are enough smaller segments whose total size adds up to the threshold.
Both the layer sizes and whether merging happens in the foreground (blocking) or background (non-blocking) are configurable.
1KB
, 10KB
, 100KB
and 1MB
are merged in the foreground. This can be configured with layer_sizes
,
either at CREATE INDEX
time or afterward with ALTER INDEX
.
layer_sizes
to 0
disables foreground merging.
10MB
, 100MB
, 1GB
, 10GB
, 100GB
, and 1TB
are merged in the background. This can be configured with
background_layer_sizes
, either at CREATE INDEX
time or afterward with ALTER INDEX
.
background_layer_sizes
to 0
disables background merging.