> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/embeddings/choosing-an-embedder.md).

# Choosing an embedder

An embedder turns text into a fixed-width `float32` vector and defines the vector space a collection lives in. Each collection binds to one embedder through its [`EmbeddingProfile`](/embeddings/embedding-profile.md), written to disk on `save()`. Picking an embedder is three decisions:

1. **Where embedding runs** — on-device in the native kernel, or in your own code.
2. **Similarity type** — lexical n-gram hashing, or dense semantic vectors.
3. **Dimension** of every vector the collection stores.

You cannot mix spaces in one collection. Changing the embedder changes the space, which means a new collection. See [EmbeddingProfile & space compatibility](/embeddings/embedding-profile.md).

Embedders are not top-level exports. Import them from `telys.embedding`:

```python
from telys.embedding import (
    AlgentaBigramEmbedder,
    AlgentaMultigramEmbedder,
    CallableEmbedder,
    EmbeddingProfile,
)
```

## The three practical choices

Three embedders ship and work in a normal install (given the native kernel):

* **`AlgentaBigramEmbedder`** — compact on-device lexical matching.
* **`AlgentaMultigramEmbedder`** — richer on-device lexical matching; the default for the MCP server and `telys mem`.
* **`CallableEmbedder`** — wraps your own function for any dense model (OpenAI, BGE, sentence-transformers) or lexical scheme.

A fourth, `AlgentaPooledEmbedder`, is listed in `telys.embedding.__all__` but resolves lazily from the signed runtime and is unavailable in a runtime-free install. See [AlgentaPooledEmbedder](/embeddings/pooled.md).

## Decision matrix

| Embedder                   | On-device?           | Network?             | Dimension source                | Best for                                                                              |
| -------------------------- | -------------------- | -------------------- | ------------------------------- | ------------------------------------------------------------------------------------- |
| `AlgentaBigramEmbedder`    | Yes — native kernel  | None                 | Native kernel (hint: 64-d)      | Compact lexical matching over short strings: keys, tags, symbol names; lowest memory. |
| `AlgentaMultigramEmbedder` | Yes — native kernel  | None                 | Native kernel (hint: 384-d)     | Richer lexical recall over prose; the MCP / `telys mem` default.                      |
| `CallableEmbedder`         | Depends on your `fn` | Depends on your `fn` | `profile.dimension` you declare | Bring-your-own dense models and services (OpenAI, BGE, sentence-transformers).        |
| `AlgentaPooledEmbedder`    | Runtime-only         | Runtime-defined      | Runtime-defined                 | Only when the signed runtime is installed; not part of the SDK surface.               |

{% hint style="info" %}
Dimensions for the Algenta embedders are **read from the native kernel** at construction time. The `64` and `384` figures are hints, not guaranteed constants — always create the collection with `emb.profile.dimension`, never a hard-coded number.
{% endhint %}

## Lexical or dense?

The Algenta embedders are **lexical**: they hash character n-grams into a bounded vector, so similarity rewards shared substrings and exact terms. They run entirely on-device, need no model download, and are strong for code, identifiers, tags, and keyword recall. They cannot match meaning worded differently — "car" and "automobile" do not match unless the characters overlap.

`CallableEmbedder` is the **dense** path: hand Telys a function that returns semantic vectors from a model you control. That gives paraphrase-aware recall at the cost of a model dependency (and possibly a network call). For the full tradeoff, see [Lexical vs dense embeddings](/understand-telys/lexical-vs-dense.md).

{% hint style="warning" %}
The Algenta embedders require the **native kernel** and raise `ImportError` without it. Any collection operation also requires the signed runtime. `CallableEmbedder` needs no kernel, but the collection it feeds still does. Run `telys runtime verify` to confirm both are present.
{% endhint %}

## Constructing each

{% tabs %}
{% tab title="Multigram (default)" icon="subscript" %}

```python
from telys import Telys
from telys.embedding import AlgentaMultigramEmbedder

engine = Telys("./memory")
emb = AlgentaMultigramEmbedder()

col = engine.create_collection(
    "notes",
    dim=emb.profile.dimension,   # kernel-sourced (hint: 384)
    partition_by="scope",
    embedder=emb,
)
col.add_texts(
    ["telys keeps memory on device"],
    ids=["n1"],
    metadata=[{"scope": "notes"}],
)
col.save()
```

{% endtab %}

{% tab title="Bigram (compact)" icon="superscript" %}

```python
from telys import Telys
from telys.embedding import AlgentaBigramEmbedder

engine = Telys("./memory")
emb = AlgentaBigramEmbedder()

col = engine.create_collection(
    "tags",
    dim=emb.profile.dimension,   # kernel-sourced (hint: 64)
    partition_by="scope",
    embedder=emb,
)
```

{% endtab %}

{% tab title="Callable (BYO dense)" icon="plug" %}

```python
import numpy as np
from telys import Telys
from telys.embedding import CallableEmbedder, EmbeddingProfile

def embed(texts):
    # call your model or service; return an (N, D) float32 array
    return np.asarray(my_model.encode(texts), dtype="float32")

profile = EmbeddingProfile(
    provider="my-org",
    model_id="text-embedding-3-small",
    model_version="1",
    dimension=1536,
)
emb = CallableEmbedder(embed, profile)

engine = Telys("./memory")
col = engine.create_collection(
    "docs",
    dim=profile.dimension,
    partition_by="scope",
    embedder=emb,
)
```

{% endtab %}
{% endtabs %}

## Reopening a collection later

`save()` records the embedder's profile with the collection. `open_collection(name)` re-attaches the embedder automatically for the Algenta providers. A `CallableEmbedder` function cannot be serialized, so register the provider first and pass the embedder back:

```python
engine.register_provider(profile.space_id(), emb)
col = engine.open_collection("docs", embedder=emb)
```

Reattach succeeds only when the profile matches — same `provider`, `model_id`, `dimension`, and pooling. That is the check `compatible_with` performs. See [Dimensions & space IDs](/embeddings/dimensions-and-space-id.md).

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>AlgentaBigramEmbedder</strong></td><td>Compact on-device lexical embeddings.</td><td><a href="/pages/E19JU21tIIqARTaX8Wp4">/pages/E19JU21tIIqARTaX8Wp4</a></td></tr><tr><td><strong>AlgentaMultigramEmbedder</strong></td><td>Richer lexical recall; the MCP default.</td><td><a href="/pages/SAD2qFASkTgunHaLDjQ6">/pages/SAD2qFASkTgunHaLDjQ6</a></td></tr><tr><td><strong>CallableEmbedder</strong></td><td>Bring your own dense model or service.</td><td><a href="/pages/Xzd0BXJbxW07haNz0M1J">/pages/Xzd0BXJbxW07haNz0M1J</a></td></tr><tr><td><strong>EmbeddingProfile</strong></td><td>How space identity and compatibility work.</td><td><a href="/pages/xwahgwdIzO1r8VFfN9l8">/pages/xwahgwdIzO1r8VFfN9l8</a></td></tr><tr><td><strong>Dimensions &#x26; space IDs</strong></td><td>Match dims and avoid mismatched spaces.</td><td><a href="/pages/MIVK3hfr15xmitNeowAz">/pages/MIVK3hfr15xmitNeowAz</a></td></tr><tr><td><strong>Recipe: OpenAI embeddings</strong></td><td>Wire a hosted dense model into Telys.</td><td><a href="/pages/osxpAbHUV1iJmzTnmdX3">/pages/osxpAbHUV1iJmzTnmdX3</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.telys.ai/embeddings/choosing-an-embedder.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.