> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/collections.md). # Collections A **collection** is the unit you work with in Telys: a named, partitioned vector store bound to one embedding space. It has a fixed dimensionality, a single partition-key column, an optional embedder, and its own directory on disk. Create one from a `Telys` handle, add and search records, then `save()` it to make it durable. ## One collection, one embedding space A collection stores fixed-width vectors of a single `dim`. Search only compares vectors in the same space, so a collection is pinned to one space for its lifetime — its `dim`, `dtype`, and (if attached) its embedder's profile. You cannot mix dimensions or embedders in one collection; use separate collections. | Property | Set at create | Meaning | | ---------------- | ------------------ | ----------------------------------------------------------- | | `name` | yes | Directory name and lookup key under the engine path | | `dim` | yes | Vector width; must match the embedder's `profile.dimension` | | `partition_by` | yes | Name of the physical partition-key column (a single column) | | `embedder` | optional | Turns text into vectors; required for the `*_texts` methods | | `dtype` | optional (`"f32"`) | On-disk vector element type | | `filter_columns` | optional (`()`) | Metadata columns you can filter on with `Eq` | | `tuner` | optional | Strategy used by `tune()` / `apply_tuning()` | ## Creating a collection `create_collection` builds a collection **in memory** and caches it on the handle. It does **not** write anything to disk until you call `save()`. ```python from telys import Telys from telys.embedding import AlgentaMultigramEmbedder db = Telys("./memory") emb = AlgentaMultigramEmbedder() # lexical; dim comes from the native kernel col = db.create_collection( "articles", dim=emb.profile.dimension, # read the space dim from the embedder partition_by="tenant_id", # a single column name embedder=emb, dtype="f32", filter_columns=("lang",), # columns you can Eq-filter later ) ``` {% hint style="info" %} Read `dim` from `emb.profile.dimension` rather than hard-coding it: the `Algenta*` embedders source their width from the native kernel, so the profile is the single source of truth. A `dim` that disagrees with the embedder makes ingest reject the vectors. {% endhint %} Bringing your own vectors instead of text? Skip the embedder — but then only the raw `add`/`upsert`/`search` methods work; the text methods raise `RuntimeError`. ## The lifecycle {% stepper %} {% step %} #### Create — in memory `create_collection(...)` returns a live `Collection` held only in memory and the handle's cache. Nothing is on disk yet, and `db.collections()` will not list it. {% endstep %} {% step %} #### Add and search Ingest records and query them. With an embedder attached you can work in text: ```python col.add_texts( ["Telys runs on-device", "IVF speeds up large collections"], ids=["a1", "a2"], metadata=[{"tenant_id": "acme", "lang": "en"}, {"tenant_id": "acme", "lang": "en"}], ) hits = col.search_text("how does search scale", top_k=5) print(hits["ids"], hits["scores"]) ``` {% endstep %} {% step %} #### Save — the commit point `save()` persists the collection and returns its directory. It writes the `id_map.npy` sidecar first, then `collection.json` **last** — that final write is the atomic commit point, so a crash mid-save never leaves a half-written collection. ```python path = col.save() # e.g. "./memory/articles" ``` `collection.json` records the collection's durable identity: `name`, `dim`, `dtype`, `key_name`, `filter_columns`, the id-map sidecar, `telys_version`, `default_target_recall`, `applied_tuner`, and the `embedder_profile`. {% endstep %} {% step %} #### Reopen — later, in a new process `open_collection` reads the collection back from disk and **auto-attaches** an embedder from the provider registry, matched by the saved profile's `model_id` / `space_id`. Register the provider first: ```python db2 = Telys("./memory") db2.register_provider("multigram", AlgentaMultigramEmbedder()) col2 = db2.open_collection("articles") # embedder re-attached automatically ``` {% endstep %} {% endstepper %} ## Listing and looking up collections `collections()` returns the sorted names of directories that contain a `collection.json` — collections that have been **saved**. A created-but-unsaved collection will not appear. ```python db.collections() # ['articles'] — only saved ones col = db["articles"] # __getitem__: returns the cached one, or opens it from disk ``` {% hint style="warning" %} `create_collection` is in-memory until `save()`. Skip `save()` and the collection vanishes when your process exits, and `collections()` never lists it. `db[name]` opens a saved collection on demand. {% endhint %} ## Where a collection lives on disk A saved collection is a self-contained directory under the engine path you gave `Telys(...)`: ``` ./memory/articles/ collection.json # the commit point + durable metadata id_map.npy # external-id ↔ internal-id map # written and owned by the runtime ``` The `collection.json` and `id_map.npy` are the SDK's format; the vector slabs are the runtime's. Copy the whole directory to move or back up a collection — no external service holds any part of it. ## Next steps
Records, IDs & payloadsWhat a single record is made of and how ids map internally./pages/25a92W1qdHq0jNswnnSy
Partitions & scope keysHow partition_by works and how to compose a scoped key./pages/Rv5cBr1GHyWvxVC7AeIF
Create a collectionThe hands-on recipe, from empty directory to first search./pages/EIWgp0wPwoYMjxP37Daw
Snapshots & persistenceSave, snapshot, and reopen collections safely./pages/3IabEB7rmGWO2uCD2t8X
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.telys.ai/understand-telys/collections.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.