> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/collections.md).

# Collections

What a Telys collection is — a named, partitioned vector store bound to a single embedding space — and its create → use → save → reopen lifecycle.

A **collection** is the unit you work with in Telys: a named, partitioned vector store bound to one embedding space. It has a fixed dimensionality, a single partition-key column, an optional embedder, and its own directory on disk. Create one from a `Telys` handle, add and search records, then `save()` it to make it durable.

## One collection, one embedding space

A collection stores fixed-width vectors of a single `dim`. Search only compares vectors in the same space, so a collection is pinned to one space for its lifetime — its `dim`, `dtype`, and (if attached) its embedder's profile. You cannot mix dimensions or embedders in one collection; use separate collections.

| Property         | Set at create      | Meaning                                                     |
| ---------------- | ------------------ | ----------------------------------------------------------- |
| `name`           | yes                | Directory name and lookup key under the engine path         |
| `dim`            | yes                | Vector width; must match the embedder's `profile.dimension` |
| `partition_by`   | yes                | Name of the physical partition-key column (a single column) |
| `embedder`       | optional           | Turns text into vectors; required for the `*_texts` methods |
| `dtype`          | optional (`"f32"`) | On-disk vector element type                                 |
| `filter_columns` | optional (`()`)    | Metadata columns you can filter on with `Eq`                |
| `tuner`          | optional           | Strategy used by `tune()` / `apply_tuning()`                |

## Creating a collection

`create_collection` builds a collection **in memory** and caches it on the handle. It does **not** write anything to disk until you call `save()`.

```python
from telys import Telys
from telys.embedding import AlgentaMultigramEmbedder

db = Telys("./memory")
emb = AlgentaMultigramEmbedder()          # lexical; dim comes from the native kernel

col = db.create_collection(
    "articles",
    dim=emb.profile.dimension,            # read the space dim from the embedder
    partition_by="tenant_id",             # a single column name
    embedder=emb,
    dtype="f32",
    filter_columns=("lang",),             # columns you can Eq-filter later
)
```

{% hint style="info" %}
Read `dim` from `emb.profile.dimension` rather than hard-coding it: the `Algenta*` embedders source their width from the native kernel, so the profile is the single source of truth. A `dim` that disagrees with the embedder makes `create_collection` raise (`ValueError: embedder dim X != collection dim Y`).
{% endhint %}

Bringing your own vectors instead of text? Skip the embedder — but then only the raw `add`/`upsert`/`search` methods work; the text methods raise `RuntimeError`.

## The lifecycle

{% stepper %}
{% step %}

#### Create — in memory

`create_collection(...)` returns a live `Collection` held only in memory and the handle's cache. Nothing is on disk yet, and `db.collections()` will not list it.
{% endstep %}

{% step %}

#### Add and search

Ingest records and query them. With an embedder attached you can work in text:

```python
col.add_texts(
    ["Telys runs on-device", "IVF speeds up large collections"],
    ids=["a1", "a2"],
    metadata=[{"tenant_id": "acme", "lang": "en"},
              {"tenant_id": "acme", "lang": "en"}],
)

hits = col.search_text("how does search scale", top_k=5)
print(hits["ids"], hits["scores"])
```

{% endstep %}

{% step %}

#### Save — the commit point

`save()` persists the collection and returns its directory. It writes the `id_map.npy` sidecar first, then `collection.json` **last** — that final write is the atomic commit point, so a crash mid-save never leaves a half-written collection.

```python
path = col.save()      # e.g. "./memory/articles"
```

`collection.json` records the collection's durable identity: `name`, `dim`, `dtype`, `key_name`, `filter_columns`, the id-map sidecar, `telys_version`, `default_target_recall`, `applied_tuner`, `lexical`, and the `embedder_profile`.
{% endstep %}

{% step %}

#### Reopen — later, in a new process

`open_collection` reads the collection back from disk and **auto-attaches** an embedder from the provider registry, matched by the saved profile's `model_id` / `space_id`. Register the provider first:

```python
db2 = Telys("./memory")
db2.register_provider("multigram", AlgentaMultigramEmbedder())
col2 = db2.open_collection("articles")     # embedder re-attached automatically
```

{% endstep %}
{% endstepper %}

## Listing and looking up collections

`collections()` returns the sorted names of directories that contain a `collection.json` — collections that have been **saved**. A created-but-unsaved collection will not appear.

```python
db.collections()          # ['articles']  — only saved ones

col = db["articles"]      # __getitem__: returns the cached one, or opens it from disk
```

{% hint style="warning" %}
`create_collection` is in-memory until `save()`. Skip `save()` and the collection vanishes when your process exits, and `collections()` never lists it. `db[name]` opens a saved collection on demand.
{% endhint %}

## Where a collection lives on disk

A saved collection is a self-contained directory under the engine path you gave `Telys(...)`:

```
./memory/articles/
  collection.json     # the commit point + durable metadata
  id_map.npy          # external-id ↔ internal-id map
  <vector slab files> # written and owned by the runtime
```

The `collection.json` and `id_map.npy` are the SDK's format; the vector slabs are the runtime's. Copy the whole directory to move or back up a collection — no external service holds any part of it.

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Records, IDs &#x26; payloads</strong></td><td>What a single record is made of and how ids map internally.</td><td><a href="https://docs.telys.ai/understand-telys/records-ids-payloads.md">Records &amp; IDs</a></td></tr><tr><td><strong>Partitions &#x26; scope keys</strong></td><td>How partition_by works and how to compose a scoped key.</td><td><a href="https://docs.telys.ai/understand-telys/partitions-scope-key.md">Partitions &amp; scopes</a></td></tr><tr><td><strong>Create a collection</strong></td><td>The hands-on recipe, from empty directory to first search.</td><td><a href="https://docs.telys.ai/working-with-data/create-a-collection.md">Create a collection</a></td></tr><tr><td><strong>Snapshots &#x26; persistence</strong></td><td>Save, snapshot, and reopen collections safely.</td><td><a href="https://docs.telys.ai/working-with-data/snapshots-and-persistence.md">Save &amp; snapshots</a></td></tr></tbody></table>
