> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/embeddings/callable-byo.md).

# Bring your own

`CallableEmbedder` turns any text-to-vector function into a Telys embedder. Point it at an OpenAI call, a local `sentence-transformers` model, or a bespoke lexical hasher — anything that returns a 2-D `float32` array — and pair it with an [`EmbeddingProfile`](/embeddings/embedding-profile.md) that declares the vector space.

```python
from telys.embedding import CallableEmbedder, EmbeddingProfile
```

## Signature

```python
CallableEmbedder(fn, profile, query_fn=None)
```

| Parameter  | Type                                          | Default    | Meaning                                                                                 |
| ---------- | --------------------------------------------- | ---------- | --------------------------------------------------------------------------------------- |
| `fn`       | `Callable[[list[str]], np.ndarray]`           | (required) | Embeds documents. Takes a list of strings, returns an `[N, D]` `float32` array.         |
| `profile`  | `EmbeddingProfile`                            | (required) | Declares the vector space: dimension, normalization, distance, provider/model identity. |
| `query_fn` | `Callable[[list[str]], np.ndarray]` \| `None` | `None`     | Optional separate embedder for queries. When `None`, `fn` is used for both.             |

`CallableEmbedder` satisfies the `EmbeddingProvider` protocol, so a collection uses it exactly like the built-in `Algenta*` embedders.

## The `fn` contract

Your function receives a list of strings and must return a NumPy array. On every call `CallableEmbedder` asserts:

* `result.ndim == 2` — one row per input, one column per dimension.
* `result.shape[1] == profile.dimension` — the width matches the declared space.

Return one row per input text, in order, as `float32`. Cast if your model emits `float64` or a Python list: `np.asarray(vecs, dtype="float32")`.

{% hint style="warning" %}
`CallableEmbedder` does **not** normalize — it stores whatever `fn` returns. With the common `distance="ip"`, inner product equals cosine similarity only when vectors are L2-normalized. Normalize inside `fn` (or return unit vectors from your model) so scores are comparable. See [EmbeddingProfile & space compatibility](/embeddings/embedding-profile.md).
{% endhint %}

## Methods

As an `EmbeddingProvider`, an instance exposes:

| Member                   | Returns            | Notes                                                          |
| ------------------------ | ------------------ | -------------------------------------------------------------- |
| `.profile`               | `EmbeddingProfile` | The space you passed in. Its `space_id()` gates reopen/attach. |
| `embed_documents(texts)` | `[N, D]` `float32` | Calls `fn`; runs the shape assertions.                         |
| `embed_queries(texts)`   | `[N, D]` `float32` | Calls `query_fn` if given, otherwise `fn`.                     |

## A minimal, offline example

This toy embedder needs no network or extra packages; it shows the mechanics only. Swap `fn` for a real model (see the recipes below) before relying on results.

```python
import numpy as np
from telys import Telys
from telys.embedding import CallableEmbedder, EmbeddingProfile

DIM = 8

def toy_embed(texts):
    out = np.zeros((len(texts), DIM), dtype="float32")
    for i, t in enumerate(texts):
        h = abs(hash(t))
        for j in range(DIM):
            out[i, j] = float((h >> j) & 1)
    norms = np.linalg.norm(out, axis=1, keepdims=True)
    return (out / np.clip(norms, 1e-9, None)).astype("float32")

profile = EmbeddingProfile(
    provider="local",
    model_id="toy-hash",
    model_version="1",
    dimension=DIM,
    normalization="l2",
    distance="ip",
)
embedder = CallableEmbedder(toy_embed, profile)

ame = Telys("./telys-byo")
col = ame.create_collection("notes", dim=DIM, partition_by="scope", embedder=embedder)
col.add_texts(
    texts=["on-device memory", "offline vector search"],
    ids=["a", "b"],
    metadata=[{"scope": "demo"}, {"scope": "demo"}],
)
col.save()

print(col.search_text("local search", top_k=2)["ids"])
```

The `dim` passed to `create_collection` must equal `profile.dimension`, or the two disagree about the space. See [Dimensions & space IDs](/embeddings/dimensions-and-space-id.md).

## Separate query embedding

Some models embed queries differently from documents — for example, instruction-tuned retrieval models prepend a task prefix to queries only. Pass `query_fn` for that asymmetry:

```python
INSTRUCTION = "Represent this sentence for searching relevant passages: "

def embed_docs(texts):
    return _model_encode(list(texts)).astype("float32")

def embed_query(texts):
    return _model_encode([INSTRUCTION + t for t in texts]).astype("float32")

embedder = CallableEmbedder(embed_docs, profile, query_fn=embed_query)
```

`add_texts` / `upsert_texts` route through `embed_documents`; `search_text` routes through `embed_queries`. Both run the same shape assertions.

## Building a matching profile

The profile identifies the space your vectors live in. Two collections are compatible only when their profiles share a `space_id()`. Set the fields that describe your model honestly:

```python
profile = EmbeddingProfile(
    provider="openai",              # who produced the vectors
    model_id="text-embedding-3-small",
    model_version="1",              # bump when the model changes
    dimension=1536,                 # MUST equal what fn returns
    dtype="float32",
    normalization="l2",             # you return unit vectors
    distance="ip",                  # inner product == cosine on unit vectors
    pooling="none",
)
```

Changing `model_id`, `dimension`, `normalization`, or `distance` produces a different `space_id()`, making the new embedder incompatible with a collection built under the old one. Full field reference: [EmbeddingProfile & space compatibility](/embeddings/embedding-profile.md).

## Common errors

<details>

<summary><code>AssertionError</code> from CallableEmbedder</summary>

`fn` returned an array whose `ndim != 2` or whose `shape[1]` differs from `profile.dimension`. Return an `[N, D]` array and make `D == profile.dimension`.

</details>

<details>

<summary><code>RuntimeError</code> — no embedder on add_texts / search_text</summary>

The collection has no embedder attached. Pass `embedder=` to `create_collection`, or register a provider so `open_collection` can auto-attach it by matching `space_id()`.

</details>

## See also

* [EmbeddingProfile & space compatibility](/embeddings/embedding-profile.md) — every profile field and how identity is computed.
* [Recipe: OpenAI embeddings](/embeddings/recipe-openai.md) — a complete cloud-backed `CallableEmbedder`.
* [Recipe: BGE / sentence-transformers](/embeddings/recipe-bge.md) — a fully offline `CallableEmbedder`.
* [Choosing an embedder](/embeddings/choosing-an-embedder.md) — when to reach for BYO vs the built-in lexical embedders.
* [Embedders API](/reference/embedders-api.md) — the full `telys.embedding` surface.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.telys.ai/embeddings/callable-byo.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.