Lowercase - ParadeDB

The lowercase filter converts all characters to lowercase, allowing for case-insensitive queries. It is enabled by default but can be configured for all tokenizers besides the literal tokenizer. To disable, append lowercase=false to the tokenizer’s arguments:

CREATE INDEX search_idx ON mock_items
USING bm25 (id, (description::pdb.simple('lowercase=false')))
WITH (key_field='id');

To demonstrate this token filter, let’s compare the output of the following two statements:

SELECT
  'Tokenize me!'::pdb.simple::text[],
  'Tokenize me!'::pdb.simple('lowercase=false')::text[];

Expected Response

     text      |     text
---------------+---------------
 {tokenize,me} | {Tokenize,me}
(1 row)

ASCII Folding

Remove Stopwords