Throughput

Several settings can be used to tune the throughput of INSERT/UPDATE/COPY statements to the BM25 index.

​

Statement Memory

Like indexing memory, the amount of memory available to INSERT/UPDATE/COPY statements and number of segments created is affected by maintenance_work_mem.

maintenance_work_mem = 16GB

Generally speaking, this is the only setting that must be tuned for optimizing these statements.

​

Statement Parallelism

paradedb.statement_parallelism controls the number of indexing threads used during INSERT/UPDATE/COPY. The default is 0, which automatically detects the “available parallelism” of the host computer.

If your typical update patterns are single-row atomic INSERTs or UPDATEs, then a value of 1 can prevent unnecessary threads from being spun up. For bulk inserts and updates, a larger value is better.

SET paradedb.statement_parallelism = 1;

​

Statement Memory Budget

paradedb.statement_memory_budget, like indexing memory defaults to maintenance_work_mem divided by the total parallelism. It sets the amount of memory to dedicate per indexing thread before the index segment needs to be written to disk. The setting is measured in megabytes.

If your typical update patterns are single-row atomic INSERTs or UPDATEs, then a value of 15MB can prevent unnecessary memory from being allocated. For bulk inserts and updates, a larger value is better.

SET paradedb.statement_memory_budget = 15;