> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/reference/embedders-api.md).

# Embedders

Reference for telys.embedding — the EmbeddingProvider base class, EmbeddingProfile, CallableEmbedder, and the Algenta bigram/multigram/pooled embedders.

Import embedders from `telys.embedding`; they are not top-level. An embedder turns text into `(N, D)` float32 vectors and carries an `EmbeddingProfile` that defines the vector space a collection lives in.

```python
from telys.embedding import (
    EmbeddingProvider,
    EmbeddingProfile,
    CallableEmbedder,
    AlgentaBigramEmbedder,
    AlgentaMultigramEmbedder,
    AlgentaPooledEmbedder,
)
```

`telys.embedding.__all__` is exactly those six names.

## EmbeddingProvider (base class)

The base class every embedder subclasses. Implement these members in a subclass to define an embedder.

| Member                   | Signature                     | Description                                   |
| ------------------------ | ----------------------------- | --------------------------------------------- |
| `profile`                | `-> EmbeddingProfile`         | The vector-space identity of this embedder.   |
| `embed_documents(texts)` | `list[str] -> [N, D] float32` | Embed stored documents.                       |
| `embed_queries(texts)`   | `list[str] -> [N, D] float32` | Embed queries. Defaults to `embed_documents`. |

```python
import numpy as np
from telys.embedding import EmbeddingProvider, EmbeddingProfile

class MyEmbedder(EmbeddingProvider):
    profile = EmbeddingProfile(provider="me", model_id="demo", model_version="1", dimension=8)
    def embed_documents(self, texts):
        return np.zeros((len(texts), 8), dtype="float32")
    # embed_queries falls back to embed_documents

assert isinstance(MyEmbedder(), EmbeddingProvider)  # plain base class — isinstance works by ordinary subclassing
```

## EmbeddingProfile

A frozen dataclass that identifies a vector space. Two embedders are interchangeable only if their profiles produce the same `space_id()`.

| Field                | Default     | Meaning                                                     |
| -------------------- | ----------- | ----------------------------------------------------------- |
| `provider`           | (required)  | Vendor / family, e.g. `"algenta"`, `"openai"`.              |
| `model_id`           | (required)  | Model name, e.g. `"multigram"`, `"text-embedding-3-small"`. |
| `model_version`      | (required)  | Version string of the model.                                |
| `dimension`          | (required)  | Vector dimension `D`.                                       |
| `dtype`              | `"float32"` | Element type.                                               |
| `normalization`      | `"l2"`      | Vector normalization.                                       |
| `distance`           | `"ip"`      | Distance metric (inner product).                            |
| `pooling`            | `"none"`    | Pooling strategy.                                           |
| `tokenizer_hash`     | `""`        | Optional tokenizer identity.                                |
| `projection_version` | `""`        | Optional projection identity.                               |
| `created`            | `""`        | Optional creation marker.                                   |

Methods:

| Method                   | Returns | Description                                           |
| ------------------------ | ------- | ----------------------------------------------------- |
| `space_id()`             | `str`   | `sha256:` + 16 hex chars over the profile identity.   |
| `compatible_with(other)` | `bool`  | `True` when `space_id()` values match.                |
| `as_dict()`              | `dict`  | Serializable form (persisted by `Collection.save()`). |

```python
from telys.embedding import EmbeddingProfile

p = EmbeddingProfile(provider="openai", model_id="text-embedding-3-small",
                     model_version="1", dimension=1536)
print(p.space_id())          # e.g. "sha256:1a2b3c4d5e6f7a8b"
print(p.compatible_with(p))  # True
```

## CallableEmbedder

Wrap any function `list[str] -> [N, D] float32` as an embedder — the bring-your-own path for OpenAI, BGE, sentence-transformers, or a bespoke function. Dense or lexical, whatever your function returns.

```python
CallableEmbedder(fn, profile, query_fn=None)
```

| Parameter  | Description                                                                                 |
| ---------- | ------------------------------------------------------------------------------------------- |
| `fn`       | `list[str] -> [N, D] float32`. Used for documents (and queries unless `query_fn` is given). |
| `profile`  | The `EmbeddingProfile` for the space `fn` produces.                                         |
| `query_fn` | Optional separate function for queries.                                                     |

Each call asserts the output has `ndim == 2` and `shape[1] == profile.dimension`.

```python
import numpy as np
from telys.embedding import CallableEmbedder, EmbeddingProfile

def embed(texts):
    return np.random.rand(len(texts), 1536).astype("float32")

profile = EmbeddingProfile(provider="openai", model_id="text-embedding-3-small",
                           model_version="1", dimension=1536)
embedder = CallableEmbedder(embed, profile)
```

## Algenta embedders

The Algenta embedders are lexical hashers implemented in the native kernel. They take **no** `space_id`/`model_id` argument — space identity is computed from the profile.

{% hint style="warning" %}
`AlgentaBigramEmbedder` and `AlgentaMultigramEmbedder` need the native kernel; instantiating without it raises `ImportError`. Force the kernel path with `$TELYS_KERNEL` (then legacy `$AME_KERNEL`). `dimension` is read from the kernel — the numbers below are hints, not guarantees.
{% endhint %}

### AlgentaBigramEmbedder

```python
AlgentaBigramEmbedder(*, max_bytes=1 << 20)
```

Lexical bigram-hash embedder. Profile: `provider="algenta"`, `model_id="bigram"`, `normalization="l2"`, `distance="ip"`, `pooling="bigram-hash"`. Dimension is read from the kernel (server hint: 64-d).

### AlgentaMultigramEmbedder

```python
AlgentaMultigramEmbedder(*, max_bytes=1 << 20)
```

Lexical embedder that concatenates `[unigram | bigram | trigram]`, each L2-normalized blockwise. Profile: `model_id="multigram"`, `pooling="multigram-hash"`. Dimension is read from the kernel (MCP default hint: 384-d). This is the default embedder for MCP and `telys mem`.

```python
from telys import Telys
from telys.embedding import AlgentaMultigramEmbedder

ame = Telys("./memory")
embedder = AlgentaMultigramEmbedder()
col = ame.create_collection("notes", dim=embedder.profile.dimension,
                            partition_by="scope", embedder=embedder)
```

### AlgentaPooledEmbedder

```python
AlgentaPooledEmbedder(...)
```

Not implemented in the SDK. It resolves lazily from the runtime and raises `ImportError` without the runtime installed. Not available in a runtime-free install.

## Choosing among them

| Embedder                   | Kind             | Needs         | Typical use                                           |
| -------------------------- | ---------------- | ------------- | ----------------------------------------------------- |
| `AlgentaMultigramEmbedder` | lexical          | native kernel | Default; strong NL→code and keyword recall on-device. |
| `AlgentaBigramEmbedder`    | lexical          | native kernel | Smaller, faster lexical space.                        |
| `AlgentaPooledEmbedder`    | (runtime-only)   | full runtime  | Advanced; not in the thin SDK.                        |
| `CallableEmbedder`         | dense or lexical | your model    | Bring OpenAI/BGE or any function.                     |

## See also

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Choosing an embedder</strong></td><td>Pick the right space for your data.</td><td><a href="https://docs.telys.ai/embeddings/choosing-an-embedder.md">Choosing an embedder</a></td></tr><tr><td><strong>EmbeddingProfile &#x26; space compatibility</strong></td><td>How space_id gates compatibility.</td><td><a href="https://docs.telys.ai/embeddings/embedding-profile.md">Embedding profiles</a></td></tr><tr><td><strong>CallableEmbedder (bring your own)</strong></td><td>Wrap OpenAI, BGE, or any function.</td><td><a href="https://docs.telys.ai/embeddings/callable-byo.md">Bring your own</a></td></tr><tr><td><strong>Dimension mismatches</strong></td><td>Fix shape and space errors.</td><td><a href="https://docs.telys.ai/troubleshooting/dimension-mismatches.md">Dimension mismatches</a></td></tr></tbody></table>
