> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/records-ids-payloads.md).

# Records & IDs

A **record** is the atom of a Telys collection. Each is four things: an *external id* you choose, a *vector* of `dim` floats, an optional *metadata payload*, and a *partition key* that places it in a scope. This page covers that anatomy and how `add`, `upsert`, and `update` differ.

## Anatomy of a record

| Part          | Type                     | Notes                                                                     |
| ------------- | ------------------------ | ------------------------------------------------------------------------- |
| External id   | `str`                    | Your stable key (e.g. `"doc-42"`). Unique within the collection.          |
| Vector        | `dim` floats (`float32`) | Either passed directly (`add`) or produced by the embedder (`add_texts`). |
| Metadata      | `dict` (optional)        | Free-form payload; supplies partition-key and `filter_columns` values.    |
| Partition key | value in metadata        | Read from the metadata field named by `partition_by`.                     |

The vector is the only required companion to the id — everything ranks on vector similarity. Metadata supplies the partition-key value and any `filter_columns` values: a collection created with `partition_by="tenant_id"` and `filter_columns=("lang",)` expects each record's metadata to carry `tenant_id` and `lang`.

```python
# One record, added as text (the embedder turns the text into the vector):
col.add_texts(
    ["Telys keeps memory on-device"],
    ids=["doc-42"],
    metadata=[{"tenant_id": "acme", "lang": "en"}],
)
```

## External ids vs internal ids

You address records by your string ids. The runtime uses dense internal ids for speed and compact storage. The SDK keeps the mapping in `id_map.npy` — written by `save()`, loaded by `open_collection`. You never touch internal ids; every method takes and returns your external ids.

```
your world          Telys SDK (id_map.npy)        runtime
"doc-42"     ◄──────────►   0
"doc-43"     ◄──────────►   1        ──►   vector slabs + IVF index
"note-x"     ◄──────────►   2
```

Two convenience operators work off that map:

```python
len(col)            # number of live external ids in the collection
"doc-42" in col     # True if that external id exists
```

## Add vs upsert vs update

The three write families differ only in how they treat ids that already exist. The right one prevents both silent overwrites and unexpected `KeyError`s.

| Method                    | New id  | Existing id       | Needs embedder      |
| ------------------------- | ------- | ----------------- | ------------------- |
| `add` / `add_texts`       | inserts | raises `KeyError` | `add_texts` only    |
| `upsert` / `upsert_texts` | inserts | replaces          | `upsert_texts` only |
| `update_texts`            | raises  | replaces          | yes                 |

* **`add` / `add_texts`** — insert **new rows only**; a pre-existing id raises `KeyError`. Use it when a collision should surface as a bug. (`add_texts` also raises `RuntimeError` without an embedder.)
* **`upsert` / `upsert_texts`** — insert new rows and replace existing ones. Idempotent re-ingestion of a changing source.
* **`update_texts`** — replace **existing rows only**; never creates. Use it when re-embedding content whose ids you already hold.

```python
col.add_texts(["v1"], ids=["m1"], metadata=[{"tenant_id": "acme"}])   # inserts m1
col.upsert_texts(["v2"], ids=["m1"], metadata=[{"tenant_id": "acme"}]) # replaces m1
col.update_texts(["v3"], ids=["m1"], metadata=[{"tenant_id": "acme"}]) # replaces m1
```

Each write method returns the list of external ids it wrote. Raw-vector variants (`add`, `upsert`) take an `(n, dim)` float32 array and never need an embedder.

{% hint style="warning" %}
`add` is strict on purpose: re-running it over existing ids raises `KeyError`, not a silent overwrite — use `upsert` for re-runnable ingestion. `update_texts` is the mirror: it refuses to create, so a typo in an id raises rather than quietly inserting a stray record.
{% endhint %}

## Reading payloads back

Search returns external ids and scores. Ask for the metadata payload alongside each result with `with_metadata=True`:

```python
from telys import Eq

res = col.search_text("on-device memory", top_k=5, with_metadata=True)
res["ids"]        # ['doc-42', ...]
res["scores"]     # [0.83, ...]
res["metadata"]   # [{'tenant_id': 'acme', 'lang': 'en'}, ...]
```

To narrow which records are eligible, filter a `filter_columns` field with `Eq` (the only filter class in Telys — a single column/value equality):

```python
res = col.search_text("on-device memory", where=Eq("lang", "en"), with_metadata=True)
```

List the live ids in a scope without searching:

```python
col.ids(where=Eq("tenant_id", "acme"))   # -> ['doc-42', 'doc-43', ...]
```

{% hint style="info" %}
Only columns you declared in `filter_columns` (and the `partition_by` column) are queryable. `Eq` is equality on a single column — there is no `And`/`Or`/`Not`, no ranges, and no comparison operators anywhere in Telys.
{% endhint %}

## Deleting records

`delete(ids)` removes records by external id and quietly ignores ids that are not present, so it is safe to call defensively:

```python
col.delete(["doc-42", "does-not-exist"])   # removes doc-42, skips the rest
```

Deletes are logical until you reclaim space; `compact()` rewrites the store to drop deleted rows. See [Versioning, soft-delete & compaction](/understand-telys/versioning-soft-delete-compaction.md).

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Add &#x26; upsert records</strong></td><td>The hands-on recipe for inserting and replacing records.</td><td><a href="/pages/673hmQEbZWacJ4KiSdnW">/pages/673hmQEbZWacJ4KiSdnW</a></td></tr><tr><td><strong>Metadata &#x26; filtering</strong></td><td>How payloads, filter_columns, and Eq work together.</td><td><a href="/pages/mCyhYY8XvfpJT4RScC5i">/pages/mCyhYY8XvfpJT4RScC5i</a></td></tr><tr><td><strong>Partitions &#x26; scope keys</strong></td><td>How the partition key places each record in a scope.</td><td><a href="/pages/Rv5cBr1GHyWvxVC7AeIF">/pages/Rv5cBr1GHyWvxVC7AeIF</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.telys.ai/understand-telys/records-ids-payloads.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.