This is a beta feature available in versions
0.25.0 and above. See How
Vector Search Works for background.Make sure the pgvector extension is
installed first. ParadeDB uses pgvector’s vector types, but not its HNSW or
IVF indexes.
vector type alongside your text and other columns. This lets you combine vector search with full text search and filters in a single index,
which can significantly improve latency/recall for selective queries.
Create the Index
Themock_items table comes with an embedding column of type vector(8) populated with sample embeddings.
In this example, embedding is added to the ParadeDB index with cosine similarity as the distance function.
vector_l2_ops for L2 distance, vector_ip_ops for inner product, or vector_cosine_ops for cosine distance.
Only pgvector’s
vector type is supported. The halfvec, sparsevec, and
bit types are not yet indexable.If you track index build progress with
pg_stat_progress_create_index,
you may notice progress appear to “stop” at intervals. This is expected:
vectors are clustered with k-means at these points, which is computationally
expensive.Index Options
ParadeDB uses a SPANN-style vector index, which is similar to an IVF index but with additional structures to improve recall and latency, especially over large datasets. The followingWITH options control how vectors are clustered and indexed. All are set at index build time and apply to every vector field in the index. An example:
Vectors are clustered by proximity, and a centroid is a cluster’s
representative vector. This setting controls the number of centroids to build,
as a fraction of the number of indexed vectors (
num_centroids = centroid_ratio * num_vectors). More centroids produce smaller clusters,
improving recall at the cost of a slower, more memory-intensive build. Must be
between 0.000001 and 1.0.default:32
The number of vectors sampled per centroid to train the k-means clustering at
build time. Higher values yield better-quality centroids at the cost of a
slower build. Must be between
1 and 100000.default:1
The number of clusters each vector is written into: its primary cluster plus
up to
cluster_replication - 1 next-nearest clusters. Higher values can
improve recall for filtered queries at the cost of a larger index. The default
of 1 disables replication.