Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

BGE recipe

Integration recipe — wrap a local BGE / sentence-transformers model in a CallableEmbedder for fully offline dense search, with no API keys and no network.

Wrap a local sentence-transformers BGE model in a CallableEmbedder. After the weights download once, the whole pipeline — embedding, indexing, and search — runs offline: no API key, no per-call cost, nothing leaves the host.

This recipe uses BAAI/bge-base-en-v1.5, which is 768-dimensional. Always match your profile's dimension to the model you load: bge-small-en-v1.5 is 384-d and bge-large-en-v1.5 is 1024-d.

Prerequisites

pip install telys sentence-transformers numpy
telys login            # free telys_developer tier
telys runtime verify   # confirm the signed runtime is active

The first SentenceTransformer(...) call downloads the weights; later runs load them from cache and need no network.

The full script

import numpy as np
from sentence_transformers import SentenceTransformer
from telys import Telys
from telys.embedding import CallableEmbedder, EmbeddingProfile

MODEL = "BAAI/bge-base-en-v1.5"
DIM = 768
model = SentenceTransformer(MODEL)

# BGE v1.5 recommends prefixing the QUERY (not documents) with a retrieval instruction.
INSTRUCTION = "Represent this sentence for searching relevant passages: "

def embed_docs(texts):
    return model.encode(list(texts), normalize_embeddings=True).astype("float32")

def embed_query(texts):
    prompted = [INSTRUCTION + t for t in texts]
    return model.encode(prompted, normalize_embeddings=True).astype("float32")

profile = EmbeddingProfile(
    provider="bge",
    model_id=MODEL,
    model_version="1.5",
    dimension=DIM,
    normalization="l2",                 # normalize_embeddings=True returns unit vectors
    distance="ip",                      # inner product == cosine on unit vectors
)
embedder = CallableEmbedder(embed_docs, profile, query_fn=embed_query)

ame = Telys("./telys-bge")
col = ame.create_collection("docs", dim=DIM, partition_by="scope", embedder=embedder,
                            filter_columns=("scope",))  # declare "scope" to read it back via with_metadata

col.add_texts(
    texts=[
        "Telys runs entirely on-device.",
        "Vectors are stored locally and never uploaded.",
        "The signed runtime is verified before it loads.",
    ],
    ids=["a", "b", "c"],
    metadata=[{"scope": "demo"}, {"scope": "demo"}, {"scope": "demo"}],
)
col.save()

hits = col.search_text("is my data uploaded anywhere?", top_k=2, with_metadata=True)
for doc_id, score, meta in zip(hits["ids"], hits["scores"], hits["metadata"]):
    print(f"{doc_id}  {score:.3f}  {meta}")

What you will see

The two nearest documents, ranked by score. Because query_fn prepends the BGE instruction, search_text embeds the query differently from the stored documents while both land in the same 768-d space:

Scores depend on the model, but the ordering is stable. normalize_embeddings=True makes each vector unit length, so distance="ip" behaves as cosine similarity.

CallableEmbedder never normalizes for you — it stores exactly what your function returns. If you drop normalize_embeddings=True, either normalize inside the function or set normalization="none" and distance="ip" and accept raw inner-product scores.

Reopen later

Next steps

Next: compare this with a hosted API in the OpenAI recipe.

Last updated

Was this helpful?