> ## Documentation Index
> Fetch the complete documentation index at: https://docs.paradedb.com/llms.txt
> Use this file to discover all available pages before exploring further.

# How Token Filters Work

> Token filters apply additional processing to tokens like lowercasing or stemming

After a [tokenizer](/documentation/tokenizers/overview) splits up text into tokens, token filters
apply additional processing to each token. Common examples include [stemming](/documentation/token-filters/stemming)
to reduce words to their root form, or [ASCII folding](/documentation/token-filters/ascii-folding) to remove accents.

Token filters can be added to any tokenizer besides the [literal](/documentation/tokenizers/available-tokenizers/literal) tokenizer, which by definition
must preserve the source text exactly.

To add a token filter to a tokenizer, append a configuration string to the argument list:

```sql theme={null}
CREATE INDEX search_idx ON mock_items
USING bm25 (id, (description::pdb.simple('stemmer=english', 'ascii_folding=true')))
WITH (key_field='id');
```