> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/embeddings-and-spaces.md). # Embeddings & spaces An **embedder** turns text into a fixed-length vector of `float32` numbers. Telys compares vectors, never raw text — and two vectors are comparable only if they came from the same embedder configured the same way. Telys makes that explicit with a **space**: a stable identity for "vectors produced by this exact embedder". A collection is bound to one space at creation, and every later read is checked against it. ## An embedder produces vectors in a space Every embedder exposes a profile and returns a `(N, D)` float32 array — one row per input text: ```python from telys.embedding import AlgentaMultigramEmbedder emb = AlgentaMultigramEmbedder() # on-device lexical embedder vecs = emb.embed_documents(["def add(a, b): return a + b"]) print(vecs.shape) # (1, D) — D is the embedder's dimension print(emb.profile.dimension) # D, sourced from the native kernel ``` `D` is the embedder's **dimension** — a property of the embedder, not a number you choose. The `Algenta*` embedders read it from the native kernel (bigram and multigram are 64 and 384 as hints, not guarantees). The collection, the on-disk slab, and every query vector must agree on it. {% hint style="info" %} The `Algenta*` embedders require the native kernel and raise `ImportError` without it. Any embedder still needs the signed runtime to run a collection operation. See [On-device & offline-first](/understand-telys/on-device-offline.md). {% endhint %} ## EmbeddingProfile: the identity of a space `EmbeddingProfile` is a frozen dataclass describing *how* vectors were made — the fields that decide whether two sets of vectors are comparable: ```python from telys.embedding import EmbeddingProfile profile = EmbeddingProfile( provider="openai", model_id="text-embedding-3-small", model_version="1", dimension=1536, normalization="l2", # default distance="ip", # default — inner product on L2-normalized vectors == cosine pooling="none", # default ) print(profile.space_id()) # e.g. "sha256:9f2a1c4b7e0d3a56" ``` `space_id()` hashes the profile's **identity fields** — provider, model\_id, model\_version, dimension, normalization, distance, pooling (plus `tokenizer_hash` and `projection_version` when set) — into a short `sha256:`-prefixed digest. Two profiles are in the same space **iff** their `space_id()` values are equal: ```python a.compatible_with(b) # True iff a.space_id() == b.space_id() ``` Non-identity fields such as `created` are metadata and do **not** change the space. Every field that affects the geometry of the vectors does: | Profile field | Example | Part of the space identity | | --------------------------------------- | ------------------ | -------------------------- | | `provider` | `"algenta"` | yes | | `model_id` | `"multigram"` | yes | | `model_version` | `"1"` | yes | | `dimension` | `384` | yes | | `normalization` | `"l2"` | yes | | `distance` | `"ip"` | yes | | `pooling` | `"multigram-hash"` | yes | | `tokenizer_hash` / `projection_version` | `""` | yes (when set) | | `created` | timestamp | no (metadata only) | Change any identity field — a new model, a different dimension, a switch of distance — and you have a **new space**. Vectors from the old space are not comparable to the new one. ## A collection is bound to one space When you create a collection you pass a `dim` and an `embedder`. The `dim` must equal `embedder.profile.dimension` — Telys stores vectors in a fixed-width slab and rejects a mismatch: ```python from telys import Telys from telys.embedding import AlgentaMultigramEmbedder db = Telys("./memory") emb = AlgentaMultigramEmbedder() col = db.create_collection( "notes", dim=emb.profile.dimension, # must match the embedder's dimension partition_by="scope", embedder=emb, ) col.add_texts(["first note"], ids=["n1"], metadata=[{}]) col.save() # persists the embedder_profile into collection.json ``` `save()` writes the embedder's profile into `collection.json`. That stored profile — and its `space_id()` — is the collection's permanent contract: every later vector and query against `notes` lives in that one space. {% hint style="warning" %} `create_collection` builds and caches the collection in memory but does **not** touch disk until you call `col.save()`. Only saved collections appear in `db.collections()`. {% endhint %} ## Reopening validates space compatibility Reopening reads the stored profile and checks the embedder you attach belongs to the same space. Pass no embedder and Telys resolves one from the provider registry by matching `model_id` / `space_id`: ```python db2 = Telys("./memory") db2.register_provider("multigram", AlgentaMultigramEmbedder()) col = db2.open_collection("notes") # embedder resolved from the registry by space res = col.search_text("first", top_k=5) ``` If the embedder's `space_id()` does **not** match the stored profile, Telys refuses to attach it: the on-disk vectors were written in the original space, so a different embedder would produce meaningless scores. This check stops a silently-changed-model bug from degrading recall. ```python # Passing a different-space embedder to open_collection is rejected on purpose: from telys.embedding import AlgentaBigramEmbedder db2.open_collection("notes", embedder=AlgentaBigramEmbedder()) # different space_id -> refused ``` ## Why this matters Silent space mismatches are the most common way a vector store goes wrong: someone bumps a model version or changes dimensions, and every later score is subtly meaningless. Telys makes the space a hashed identity so the mismatch is caught at open time, not weeks later as mysteriously bad results. Moving to a new model is a new space — create a new collection and re-embed rather than mixing spaces in one collection. ## Next steps
EmbeddingProfile & space compatibilityThe full profile field reference and space_id() rules./pages/xwahgwdIzO1r8VFfN9l8
Choosing an embedderPick the right embedder for your data and privacy needs./pages/XuNwRcXB5hZbgTGaTAau
Dimensions & space IDsHow dimension is sourced and how space IDs are derived./pages/MIVK3hfr15xmitNeowAz
Next: read [Lexical vs dense embeddings](/understand-telys/lexical-vs-dense.md) to choose what actually produces your vectors. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.telys.ai/understand-telys/embeddings-and-spaces.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.