> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/reference/embedders-api.md).

# Embedders

Import embedders from `telys.embedding`; they are not top-level. An embedder turns text into `(N, D)` float32 vectors and carries an `EmbeddingProfile` that defines the vector space a collection lives in.

```python
from telys.embedding import (
    EmbeddingProvider,
    EmbeddingProfile,
    CallableEmbedder,
    AlgentaBigramEmbedder,
    AlgentaMultigramEmbedder,
    AlgentaPooledEmbedder,
)
```

`telys.embedding.__all__` is exactly those six names.

## EmbeddingProvider (protocol)

The structural interface every embedder satisfies. Any object with these members is an embedder.

| Member                   | Signature                     | Description                                   |
| ------------------------ | ----------------------------- | --------------------------------------------- |
| `profile`                | `-> EmbeddingProfile`         | The vector-space identity of this embedder.   |
| `embed_documents(texts)` | `list[str] -> [N, D] float32` | Embed stored documents.                       |
| `embed_queries(texts)`   | `list[str] -> [N, D] float32` | Embed queries. Defaults to `embed_documents`. |

```python
import numpy as np
from telys.embedding import EmbeddingProvider, EmbeddingProfile

class MyEmbedder(EmbeddingProvider):
    profile = EmbeddingProfile(provider="me", model_id="demo", model_version="1", dimension=8)
    def embed_documents(self, texts):
        return np.zeros((len(texts), 8), dtype="float32")
    # embed_queries falls back to embed_documents

assert isinstance(MyEmbedder(), EmbeddingProvider)  # runtime-checkable protocol
```

## EmbeddingProfile

A frozen dataclass that identifies a vector space. Two embedders are interchangeable only if their profiles produce the same `space_id()`.

| Field                | Default     | Meaning                                                     |
| -------------------- | ----------- | ----------------------------------------------------------- |
| `provider`           | (required)  | Vendor / family, e.g. `"algenta"`, `"openai"`.              |
| `model_id`           | (required)  | Model name, e.g. `"multigram"`, `"text-embedding-3-small"`. |
| `model_version`      | (required)  | Version string of the model.                                |
| `dimension`          | (required)  | Vector dimension `D`.                                       |
| `dtype`              | `"float32"` | Element type.                                               |
| `normalization`      | `"l2"`      | Vector normalization.                                       |
| `distance`           | `"ip"`      | Distance metric (inner product).                            |
| `pooling`            | `"none"`    | Pooling strategy.                                           |
| `tokenizer_hash`     | `""`        | Optional tokenizer identity.                                |
| `projection_version` | `""`        | Optional projection identity.                               |
| `created`            | `""`        | Optional creation marker.                                   |

Methods:

| Method                   | Returns | Description                                           |
| ------------------------ | ------- | ----------------------------------------------------- |
| `space_id()`             | `str`   | `sha256:` + 16 hex chars over the profile identity.   |
| `compatible_with(other)` | `bool`  | `True` when `space_id()` values match.                |
| `as_dict()`              | `dict`  | Serializable form (persisted by `Collection.save()`). |

```python
from telys.embedding import EmbeddingProfile

p = EmbeddingProfile(provider="openai", model_id="text-embedding-3-small",
                     model_version="1", dimension=1536)
print(p.space_id())          # e.g. "sha256:1a2b3c4d5e6f7a8b"
print(p.compatible_with(p))  # True
```

## CallableEmbedder

Wrap any function `list[str] -> [N, D] float32` as an embedder — the bring-your-own path for OpenAI, BGE, sentence-transformers, or a bespoke function. Dense or lexical, whatever your function returns.

```python
CallableEmbedder(fn, profile, query_fn=None)
```

| Parameter  | Description                                                                                 |
| ---------- | ------------------------------------------------------------------------------------------- |
| `fn`       | `list[str] -> [N, D] float32`. Used for documents (and queries unless `query_fn` is given). |
| `profile`  | The `EmbeddingProfile` for the space `fn` produces.                                         |
| `query_fn` | Optional separate function for queries.                                                     |

Each call asserts the output has `ndim == 2` and `shape[1] == profile.dimension`.

```python
import numpy as np
from telys.embedding import CallableEmbedder, EmbeddingProfile

def embed(texts):
    return np.random.rand(len(texts), 1536).astype("float32")

profile = EmbeddingProfile(provider="openai", model_id="text-embedding-3-small",
                           model_version="1", dimension=1536)
embedder = CallableEmbedder(embed, profile)
```

## Algenta embedders

The Algenta embedders are lexical hashers implemented in the native kernel. They take **no** `space_id`/`model_id` argument — space identity is computed from the profile.

{% hint style="warning" %}
`AlgentaBigramEmbedder` and `AlgentaMultigramEmbedder` need the native kernel; instantiating without it raises `ImportError`. Force the kernel path with `$TELYS_KERNEL` (then legacy `$AME_KERNEL`). `dimension` is read from the kernel — the numbers below are hints, not guarantees.
{% endhint %}

### AlgentaBigramEmbedder

```python
AlgentaBigramEmbedder(*, max_bytes=1 << 20)
```

Lexical bigram-hash embedder. Profile: `provider="algenta"`, `model_id="bigram"`, `normalization="l2"`, `distance="ip"`, `pooling="bigram-hash"`. Dimension is read from the kernel (server hint: 64-d).

### AlgentaMultigramEmbedder

```python
AlgentaMultigramEmbedder(*, max_bytes=1 << 20)
```

Lexical embedder that concatenates `[unigram | bigram | trigram]`, each L2-normalized blockwise. Profile: `model_id="multigram"`, `pooling="multigram-hash"`. Dimension is read from the kernel (MCP default hint: 384-d). This is the default embedder for MCP and `telys mem`.

```python
from telys.embedding import AlgentaMultigramEmbedder

embedder = AlgentaMultigramEmbedder()
col = ame.create_collection("notes", dim=embedder.profile.dimension,
                            partition_by="scope", embedder=embedder)
```

### AlgentaPooledEmbedder

```python
AlgentaPooledEmbedder(...)
```

Not implemented in the SDK. It resolves lazily from the runtime and raises `ImportError` without the runtime installed. Not available in a runtime-free install.

## Choosing among them

| Embedder                   | Kind             | Needs         | Typical use                                           |
| -------------------------- | ---------------- | ------------- | ----------------------------------------------------- |
| `AlgentaMultigramEmbedder` | lexical          | native kernel | Default; strong NL→code and keyword recall on-device. |
| `AlgentaBigramEmbedder`    | lexical          | native kernel | Smaller, faster lexical space.                        |
| `AlgentaPooledEmbedder`    | (runtime-only)   | full runtime  | Advanced; not in the thin SDK.                        |
| `CallableEmbedder`         | dense or lexical | your model    | Bring OpenAI/BGE or any function.                     |

## See also

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Choosing an embedder</strong></td><td>Pick the right space for your data.</td><td><a href="/pages/XuNwRcXB5hZbgTGaTAau">/pages/XuNwRcXB5hZbgTGaTAau</a></td></tr><tr><td><strong>EmbeddingProfile &#x26; space compatibility</strong></td><td>How space_id gates compatibility.</td><td><a href="/pages/xwahgwdIzO1r8VFfN9l8">/pages/xwahgwdIzO1r8VFfN9l8</a></td></tr><tr><td><strong>CallableEmbedder (bring your own)</strong></td><td>Wrap OpenAI, BGE, or any function.</td><td><a href="/pages/Xzd0BXJbxW07haNz0M1J">/pages/Xzd0BXJbxW07haNz0M1J</a></td></tr><tr><td><strong>Dimension mismatches</strong></td><td>Fix shape and space errors.</td><td><a href="/pages/cLf0cG2Rm2TZeyApGR9I">/pages/cLf0cG2Rm2TZeyApGR9I</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.telys.ai/reference/embedders-api.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.