> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/records-ids-payloads.md).

# Records & IDs

Anatomy of a Telys record — your external id, a vector, an optional metadata payload, and a partition key — plus add vs upsert vs update semantics.

A **record** is the atom of a Telys collection. Each is four things: an *external id* you choose, a *vector* of `dim` floats, an optional *metadata payload*, and a *partition key* that places it in a scope. This page covers that anatomy and how `add`, `upsert`, and `update` differ.

## Anatomy of a record

| Part          | Type                     | Notes                                                                     |
| ------------- | ------------------------ | ------------------------------------------------------------------------- |
| External id   | `str`                    | Your stable key (e.g. `"doc-42"`). Unique within the collection.          |
| Vector        | `dim` floats (`float32`) | Either passed directly (`add`) or produced by the embedder (`add_texts`). |
| Metadata      | `dict` (optional)        | Free-form payload; supplies partition-key and `filter_columns` values.    |
| Partition key | value in metadata        | Read from the metadata field named by `partition_by`.                     |

The vector is the only required companion to the id — everything ranks on vector similarity. Metadata supplies the partition-key value and any `filter_columns` values: a collection created with `partition_by="tenant_id"` and `filter_columns=("lang",)` expects each record's metadata to carry `tenant_id` and `lang`.

```python
# One record, added as text (the embedder turns the text into the vector):
col.add_texts(
    ["Telys keeps memory on-device"],
    ids=["doc-42"],
    metadata=[{"tenant_id": "acme", "lang": "en"}],
)
```

## External ids vs internal ids

You address records by your string ids. The runtime uses dense internal ids for speed and compact storage. The SDK keeps the mapping in `id_map.npy` — written by `save()`, loaded by `open_collection`. You never touch internal ids; every method takes and returns your external ids.

```
your world          Telys SDK (id_map.npy)        runtime
"doc-42"     ◄──────────►   0
"doc-43"     ◄──────────►   1        ──►   vector slabs + IVF index
"note-x"     ◄──────────►   2
```

Two convenience operators work off that map:

```python
len(col)            # number of live external ids in the collection
"doc-42" in col     # True if that external id exists
```

## Add vs upsert vs update

The three write families differ only in how they treat ids that already exist. The right one prevents both silent overwrites and unexpected `KeyError`s.

| Method                    | New id  | Existing id       | Needs embedder      |
| ------------------------- | ------- | ----------------- | ------------------- |
| `add` / `add_texts`       | inserts | raises `KeyError` | `add_texts` only    |
| `upsert` / `upsert_texts` | inserts | replaces          | `upsert_texts` only |
| `update_texts`            | raises  | replaces          | yes                 |

* **`add` / `add_texts`** — insert **new rows only**; a pre-existing id raises `KeyError`. Use it when a collision should surface as a bug. (`add_texts` also raises `RuntimeError` without an embedder.)
* **`upsert` / `upsert_texts`** — insert new rows and replace existing ones. Idempotent re-ingestion of a changing source.
* **`update_texts`** — replace **existing rows only**; never creates. Use it when re-embedding content whose ids you already hold.

```python
col.add_texts(["v1"], ids=["m1"], metadata=[{"tenant_id": "acme"}])   # inserts m1
col.upsert_texts(["v2"], ids=["m1"], metadata=[{"tenant_id": "acme"}]) # replaces m1
col.update_texts(["v3"], ids=["m1"], metadata=[{"tenant_id": "acme"}]) # replaces m1
```

Each write method returns the list of external ids it wrote. Raw-vector variants (`add`, `upsert`) take an `(n, dim)` float32 array and never need an embedder.

{% hint style="warning" %}
`add` is strict on purpose: re-running it over existing ids raises `KeyError`, not a silent overwrite — use `upsert` for re-runnable ingestion. `update_texts` is the mirror: it refuses to create, so a typo in an id raises rather than quietly inserting a stray record.
{% endhint %}

## Reading payloads back

Search returns external ids and scores. Ask for the metadata payload alongside each result with `with_metadata=True`:

```python
from telys import Eq

res = col.search_text("on-device memory", top_k=5, with_metadata=True)
res["ids"]        # ['doc-42', ...]
res["scores"]     # [0.83, ...]
res["metadata"]   # [{'tenant_id': 'acme', 'lang': 'en'}, ...]
```

To narrow which records are eligible, filter a `filter_columns` field with `Eq` (the only filter class in Telys — a single column/value equality):

```python
res = col.search_text("on-device memory", where=Eq("lang", "en"), with_metadata=True)
```

List the live ids in a scope without searching:

```python
col.ids(where=Eq("tenant_id", "acme"))   # -> ['doc-42', 'doc-43', ...]
```

{% hint style="info" %}
Only columns you declared in `filter_columns` (and the `partition_by` column) are queryable. `Eq` is equality on a single column — there is no `And`/`Or`/`Not`, no ranges, and no comparison operators anywhere in Telys.
{% endhint %}

## Deleting records

`delete(ids)` removes records by external id and quietly ignores ids that are not present, so it is safe to call defensively:

```python
col.delete(["doc-42", "does-not-exist"])   # removes doc-42, skips the rest
```

Deletes are logical until you reclaim space; `compact()` rewrites the store to drop deleted rows. See [Versioning, soft-delete & compaction](https://docs.telys.ai/understand-telys/versioning-soft-delete-compaction.md).

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Add &#x26; upsert records</strong></td><td>The hands-on recipe for inserting and replacing records.</td><td><a href="https://docs.telys.ai/working-with-data/add-and-upsert.md">Add &amp; upsert</a></td></tr><tr><td><strong>Metadata &#x26; filtering</strong></td><td>How payloads, filter_columns, and Eq work together.</td><td><a href="https://docs.telys.ai/understand-telys/metadata-and-filtering.md">Metadata &amp; filtering</a></td></tr><tr><td><strong>Partitions &#x26; scope keys</strong></td><td>How the partition key places each record in a scope.</td><td><a href="https://docs.telys.ai/understand-telys/partitions-scope-key.md">Partitions &amp; scopes</a></td></tr></tbody></table>
