> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/recipes/with-openai-anthropic.md).

# OpenAI & Anthropic

Pair Telys retrieval with an OpenAI or Anthropic chat model — embed with CallableEmbedder or a local Algenta embedder, retrieve, and augment the prompt.

Telys handles retrieval; a hosted chat model handles generation. Embed and index documents in Telys, retrieve the most relevant ones for a question, and pass them as context to OpenAI or Anthropic. Two independent choices:

* **Embeddings** — a hosted embedder (OpenAI) wrapped in `CallableEmbedder`, or a local Algenta embedder that runs on-device with no API key.
* **Generation** — an OpenAI or Anthropic chat model, prompted with the retrieved context.

{% hint style="info" %}
Install with `pipx install telys`, sign in with `telys login`, then verify with `telys runtime verify`. Also install the provider SDK (`pip install openai` or `pip install anthropic`) and set its API key. See [Install Telys](https://docs.telys.ai/start-here/install.md).
{% endhint %}

## Index with an embedder

Choose where embeddings come from. A hosted embedder gives a dense space matched to the provider; a local Algenta embedder keeps everything on your machine.

{% tabs %}
{% tab title="OpenAI embeddings" icon="cloud" %}
Wrap the OpenAI embeddings endpoint in `CallableEmbedder`. Your function takes a list of strings and returns an `(n, dim)` float32 array; `EmbeddingProfile` declares the space identity so Telys rejects a query embedded in an incompatible space.

{% code title="index\_openai.py" lineNumbers="true" %}

```python
import numpy as np
from openai import OpenAI

from telys import Telys
from telys.embedding import CallableEmbedder, EmbeddingProfile

client = OpenAI()  # reads OPENAI_API_KEY from the environment


def embed(texts: list[str]) -> np.ndarray:
    resp = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return np.array([d.embedding for d in resp.data], dtype="float32")


profile = EmbeddingProfile(
    provider="openai",
    model_id="text-embedding-3-small",
    model_version="1",
    dimension=1536,          # must match what `embed` returns
    normalization="l2",      # text-embedding-3 vectors are unit length
    distance="ip",           # inner product on unit vectors == cosine
)
embedder = CallableEmbedder(embed, profile)

col = Telys("./rag-store").create_collection(
    "docs", dim=profile.dimension, partition_by="source", embedder=embedder,
    filter_columns=("text",),   # persisted + returned by search(..., with_metadata=True)
)
```

{% endcode %}
{% endtab %}

{% tab title="Local Algenta" icon="microchip" %}
No API key, no network — the multigram embedder runs entirely on-device via the native kernel. Its dimension comes from the kernel, so read it from the profile.

{% code title="index\_local.py" lineNumbers="true" %}

```python
from telys import Telys
from telys.embedding import AlgentaMultigramEmbedder

embedder = AlgentaMultigramEmbedder()

col = Telys("./rag-store").create_collection(
    "docs",
    dim=embedder.profile.dimension,   # kernel-sourced
    partition_by="source",
    embedder=embedder,
    filter_columns=("text",),   # persisted + returned by search(..., with_metadata=True)
)
```

{% endcode %}
{% endtab %}
{% endtabs %}

Ingest the same way with either embedder. Keep the source text in metadata to rebuild the prompt context later:

```python
docs = [
    {"id": "d1", "text": "Telys stores vectors on-device behind a signed runtime."},
    {"id": "d2", "text": "Collections are partitioned; scope_key composes one key."},
    {"id": "d3", "text": "search_text embeds the query, then runs the search."},
]
col.add_texts(
    [d["text"] for d in docs],
    [d["id"] for d in docs],
    [{"source": "handbook", "text": d["text"]} for d in docs],
)
col.save()
```

## Retrieve and generate

Retrieval is one `search_text` call. Feed the hits to the chat model as context and instruct it to answer only from that context.

{% tabs %}
{% tab title="OpenAI" icon="cloud" %}
{% code title="ask\_openai.py" lineNumbers="true" %}

```python
from openai import OpenAI

from telys import Telys
# `embedder` is the same CallableEmbedder used at index time.

client = OpenAI()


def ask(question: str, top_k: int = 4) -> str:
    col = Telys("./rag-store").open_collection("docs", embedder=embedder)
    res = col.search_text(question, top_k=top_k, with_metadata=True)
    context = "\n".join(f"- {m['text']}" for m in res["metadata"])

    chat = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Answer using only the provided context."},
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return chat.choices[0].message.content


if __name__ == "__main__":
    print(ask("How does Telys keep vectors private?"))
```

{% endcode %}
{% endtab %}

{% tab title="Anthropic" icon="robot" %}
The Anthropic SDK returns a list of content blocks; read the text from the `text` block.

{% code title="ask\_anthropic.py" lineNumbers="true" %}

```python
from anthropic import Anthropic

from telys import Telys
# `embedder` is the same embedder used at index time (Algenta or CallableEmbedder).

client = Anthropic()  # reads ANTHROPIC_API_KEY from the environment


def ask(question: str, top_k: int = 4) -> str:
    col = Telys("./rag-store").open_collection("docs", embedder=embedder)
    res = col.search_text(question, top_k=top_k, with_metadata=True)
    context = "\n".join(f"- {m['text']}" for m in res["metadata"])

    msg = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        system="Answer using only the provided context.",
        messages=[
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return next(block.text for block in msg.content if block.type == "text")


if __name__ == "__main__":
    print(ask("How does Telys keep vectors private?"))
```

{% endcode %}
{% endtab %}
{% endtabs %}

{% hint style="warning" %}
The query-time embedder must produce the **same vector space** as the index-time one — same provider, model, and dimension. `EmbeddingProfile.space_id()` encodes that identity, and Telys raises on a mismatch instead of returning silently wrong results. Reopen with the embedder you indexed with. See [EmbeddingProfile & space compatibility](https://docs.telys.ai/embeddings/embedding-profile.md).
{% endhint %}

## What you will see

Both variants retrieve the on-device / privacy document as the top hit and produce a grounded answer:

```
Telys stores vectors on-device behind a signed native runtime, so your documents
never leave the machine — retrieval runs locally and only the assembled prompt is
sent to the chat model.
```

With OpenAI embeddings, document vectors are computed by OpenAI at index time; the local Algenta path computes them on-device. Either way, the chat model sees only the small retrieved context you assemble — not your whole corpus.

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Recipe: OpenAI embeddings</strong></td><td>The full CallableEmbedder + OpenAI embeddings walkthrough.</td><td><a href="https://docs.telys.ai/embeddings/recipe-openai.md">OpenAI recipe</a></td></tr><tr><td><strong>CallableEmbedder (bring your own)</strong></td><td>Wrap any embedding function as a Telys embedder.</td><td><a href="https://docs.telys.ai/embeddings/callable-byo.md">Bring your own</a></td></tr><tr><td><strong>RAG over a codebase</strong></td><td>Retrieval-augmented generation over source code.</td><td><a href="https://docs.telys.ai/recipes/rag-over-a-codebase.md">RAG over a codebase</a></td></tr></tbody></table>
