> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/recipes/with-openai-anthropic.md).

# OpenAI & Anthropic

Telys handles retrieval; a hosted chat model handles generation. Embed and index documents in Telys, retrieve the most relevant ones for a question, and pass them as context to OpenAI or Anthropic. Two independent choices:

* **Embeddings** — a hosted embedder (OpenAI) wrapped in `CallableEmbedder`, or a local Algenta embedder that runs on-device with no API key.
* **Generation** — an OpenAI or Anthropic chat model, prompted with the retrieved context.

{% hint style="info" %}
Install with `pipx install telys`, sign in with `telys login`, then verify with `telys runtime verify`. Also install the provider SDK (`pip install openai` or `pip install anthropic`) and set its API key. See [Install Telys](/start-here/install.md).
{% endhint %}

## Index with an embedder

Choose where embeddings come from. A hosted embedder gives a dense space matched to the provider; a local Algenta embedder keeps everything on your machine.

{% tabs %}
{% tab title="OpenAI embeddings" icon="cloud" %}
Wrap the OpenAI embeddings endpoint in `CallableEmbedder`. Your function takes a list of strings and returns an `(n, dim)` float32 array; `EmbeddingProfile` declares the space identity so Telys rejects a query embedded in an incompatible space.

{% code title="index\_openai.py" lineNumbers="true" %}

```python
import numpy as np
from openai import OpenAI

from telys import Telys
from telys.embedding import CallableEmbedder, EmbeddingProfile

client = OpenAI()  # reads OPENAI_API_KEY from the environment


def embed(texts: list[str]) -> np.ndarray:
    resp = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return np.array([d.embedding for d in resp.data], dtype="float32")


profile = EmbeddingProfile(
    provider="openai",
    model_id="text-embedding-3-small",
    model_version="1",
    dimension=1536,          # must match what `embed` returns
    normalization="l2",      # text-embedding-3 vectors are unit length
    distance="ip",           # inner product on unit vectors == cosine
)
embedder = CallableEmbedder(embed, profile)

col = Telys("./rag-store").create_collection(
    "docs", dim=profile.dimension, partition_by="source", embedder=embedder,
    filter_columns=("text",),   # persisted + returned by search(..., with_metadata=True)
)
```

{% endcode %}
{% endtab %}

{% tab title="Local Algenta" icon="microchip" %}
No API key, no network — the multigram embedder runs entirely on-device via the native kernel. Its dimension comes from the kernel, so read it from the profile.

{% code title="index\_local.py" lineNumbers="true" %}

```python
from telys import Telys
from telys.embedding import AlgentaMultigramEmbedder

embedder = AlgentaMultigramEmbedder()

col = Telys("./rag-store").create_collection(
    "docs",
    dim=embedder.profile.dimension,   # kernel-sourced
    partition_by="source",
    embedder=embedder,
    filter_columns=("text",),   # persisted + returned by search(..., with_metadata=True)
)
```

{% endcode %}
{% endtab %}
{% endtabs %}

Ingest the same way with either embedder. Keep the source text in metadata to rebuild the prompt context later:

```python
docs = [
    {"id": "d1", "text": "Telys stores vectors on-device behind a signed runtime."},
    {"id": "d2", "text": "Collections are partitioned; scope_key composes one key."},
    {"id": "d3", "text": "search_text embeds the query, then runs the search."},
]
col.add_texts(
    [d["text"] for d in docs],
    [d["id"] for d in docs],
    [{"source": "handbook", "text": d["text"]} for d in docs],
)
col.save()
```

## Retrieve and generate

Retrieval is one `search_text` call. Feed the hits to the chat model as context and instruct it to answer only from that context.

{% tabs %}
{% tab title="OpenAI" icon="cloud" %}
{% code title="ask\_openai.py" lineNumbers="true" %}

```python
from openai import OpenAI

from telys import Telys
# `embedder` is the same CallableEmbedder used at index time.

client = OpenAI()


def ask(question: str, top_k: int = 4) -> str:
    col = Telys("./rag-store").open_collection("docs", embedder=embedder)
    res = col.search_text(question, top_k=top_k, with_metadata=True)
    context = "\n".join(f"- {m['text']}" for m in res["metadata"])

    chat = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Answer using only the provided context."},
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return chat.choices[0].message.content


if __name__ == "__main__":
    print(ask("How does Telys keep vectors private?"))
```

{% endcode %}
{% endtab %}

{% tab title="Anthropic" icon="robot" %}
The Anthropic SDK returns a list of content blocks; read the text from the `text` block.

{% code title="ask\_anthropic.py" lineNumbers="true" %}

```python
from anthropic import Anthropic

from telys import Telys
# `embedder` is the same embedder used at index time (Algenta or CallableEmbedder).

client = Anthropic()  # reads ANTHROPIC_API_KEY from the environment


def ask(question: str, top_k: int = 4) -> str:
    col = Telys("./rag-store").open_collection("docs", embedder=embedder)
    res = col.search_text(question, top_k=top_k, with_metadata=True)
    context = "\n".join(f"- {m['text']}" for m in res["metadata"])

    msg = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        system="Answer using only the provided context.",
        messages=[
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"},
        ],
    )
    return next(block.text for block in msg.content if block.type == "text")


if __name__ == "__main__":
    print(ask("How does Telys keep vectors private?"))
```

{% endcode %}
{% endtab %}
{% endtabs %}

{% hint style="warning" %}
The query-time embedder must produce the **same vector space** as the index-time one — same provider, model, and dimension. `EmbeddingProfile.space_id()` encodes that identity, and Telys raises on a mismatch instead of returning silently wrong results. Reopen with the embedder you indexed with. See [EmbeddingProfile & space compatibility](/embeddings/embedding-profile.md).
{% endhint %}

## What you will see

Both variants retrieve the on-device / privacy document as the top hit and produce a grounded answer:

```
Telys stores vectors on-device behind a signed native runtime, so your documents
never leave the machine — retrieval runs locally and only the assembled prompt is
sent to the chat model.
```

With OpenAI embeddings, document vectors are computed by OpenAI at index time; the local Algenta path computes them on-device. Either way, the chat model sees only the small retrieved context you assemble — not your whole corpus.

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Recipe: OpenAI embeddings</strong></td><td>The full CallableEmbedder + OpenAI embeddings walkthrough.</td><td><a href="/pages/osxpAbHUV1iJmzTnmdX3">/pages/osxpAbHUV1iJmzTnmdX3</a></td></tr><tr><td><strong>CallableEmbedder (bring your own)</strong></td><td>Wrap any embedding function as a Telys embedder.</td><td><a href="/pages/Xzd0BXJbxW07haNz0M1J">/pages/Xzd0BXJbxW07haNz0M1J</a></td></tr><tr><td><strong>RAG over a codebase</strong></td><td>Retrieval-augmented generation over source code.</td><td><a href="/pages/3eE2RUN6Caqv3I5vLIpd">/pages/3eE2RUN6Caqv3I5vLIpd</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.telys.ai/recipes/with-openai-anthropic.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.