BGE recipe
Integration recipe — wrap a local BGE / sentence-transformers model in a CallableEmbedder for fully offline dense search, with no API keys and no network.
Wrap a local sentence-transformers BGE model in a CallableEmbedder. After the weights download once, the whole pipeline — embedding, indexing, and search — runs offline: no API key, no per-call cost, nothing leaves the host.
This recipe uses BAAI/bge-base-en-v1.5, which is 768-dimensional. Always match your profile's dimension to the model you load: bge-small-en-v1.5 is 384-d and bge-large-en-v1.5 is 1024-d.
Prerequisites
pip install telys sentence-transformers numpy
telys login # free telys_developer tier
telys runtime verify # confirm the signed runtime is activeThe first SentenceTransformer(...) call downloads the weights; later runs load them from cache and need no network.
The full script
import numpy as np
from sentence_transformers import SentenceTransformer
from telys import Telys
from telys.embedding import CallableEmbedder, EmbeddingProfile
MODEL = "BAAI/bge-base-en-v1.5"
DIM = 768
model = SentenceTransformer(MODEL)
# BGE v1.5 recommends prefixing the QUERY (not documents) with a retrieval instruction.
INSTRUCTION = "Represent this sentence for searching relevant passages: "
def embed_docs(texts):
return model.encode(list(texts), normalize_embeddings=True).astype("float32")
def embed_query(texts):
prompted = [INSTRUCTION + t for t in texts]
return model.encode(prompted, normalize_embeddings=True).astype("float32")
profile = EmbeddingProfile(
provider="bge",
model_id=MODEL,
model_version="1.5",
dimension=DIM,
normalization="l2", # normalize_embeddings=True returns unit vectors
distance="ip", # inner product == cosine on unit vectors
)
embedder = CallableEmbedder(embed_docs, profile, query_fn=embed_query)
ame = Telys("./telys-bge")
col = ame.create_collection("docs", dim=DIM, partition_by="scope", embedder=embedder,
filter_columns=("scope",)) # declare "scope" to read it back via with_metadata
col.add_texts(
texts=[
"Telys runs entirely on-device.",
"Vectors are stored locally and never uploaded.",
"The signed runtime is verified before it loads.",
],
ids=["a", "b", "c"],
metadata=[{"scope": "demo"}, {"scope": "demo"}, {"scope": "demo"}],
)
col.save()
hits = col.search_text("is my data uploaded anywhere?", top_k=2, with_metadata=True)
for doc_id, score, meta in zip(hits["ids"], hits["scores"], hits["metadata"]):
print(f"{doc_id} {score:.3f} {meta}")What you will see
The two nearest documents, ranked by score. Because query_fn prepends the BGE instruction, search_text embeds the query differently from the stored documents while both land in the same 768-d space:
Scores depend on the model, but the ordering is stable. normalize_embeddings=True makes each vector unit length, so distance="ip" behaves as cosine similarity.
Reopen later
Next steps
Next: compare this with a hosted API in the OpenAI recipe.
Last updated
Was this helpful?