> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/tuning-tradeoffs.md).

# Tuning & tradeoffs

The recall / latency / memory tradeoff behind Telys search, how exact-crossover and IVF knobs move it, and how a Tuner turns a workload into an applied TuningPlan.

Search quality is a negotiation between three quantities that pull against each other: **recall** (did you find the true nearest neighbours?), **latency** (how long the query took), and **memory** (index overhead). Telys exposes the tradeoff through a small surface — an exact-vs-approximate crossover, IVF knobs, and a default recall target — and lets a `Tuner` choose good values and hand back a `TuningPlan` you inspect before applying.

## Three quantities in tension

You cannot maximise all three at once; tuning is about picking where you sit.

| Quantity        | You raise it by…                                         | And you pay in…                      |
| --------------- | -------------------------------------------------------- | ------------------------------------ |
| Recall          | a higher `target_recall`, more IVF probes                | latency                              |
| Latency (lower) | fewer probes, smaller `top_k`, staying exact while small | recall at scale                      |
| Memory          | building an IVF index for speed                          | index overhead on top of the vectors |

The right point depends on your workload. An agent's long-term memory that fits in tens of thousands of rows wants exact, high-recall search; a multi-million-row corpus wants an IVF index tuned for latency at an acceptable recall.

## Exact vs approximate: the crossover

Below a row threshold — the **exact-crossover** — Telys answers with an exact scan of the partition: 100% recall, and fast because the set is small. Above it, Telys switches to the **IVF** (inverted-file) index: vectors are grouped into `nlist` cells and each query probes the `nprobe` closest cells instead of scanning everything, trading a sliver of recall for a large latency drop at scale.

`build_ivf(min_rows=20000, target_recall=0.98)` builds that index explicitly. The default `min_rows=20000` is the point past which an index earns its keep; `target_recall` steers how many cells a query must probe to hit the desired recall.

```python
from telys import Telys
from telys.embedding import AlgentaBigramEmbedder

db = Telys("./data")
docs = db.open_collection("docs", embedder=AlgentaBigramEmbedder())

# Build a per-partition IVF once the collection is large enough to benefit.
docs.build_ivf(min_rows=20000, target_recall=0.98)
docs.save()
```

## The recall dial

A collection carries a `default_target_recall`. Every `search`/`search_text` uses it unless a call overrides it — set the baseline once and raise the bar only for queries that need it:

```python
# Uses the collection's default_target_recall.
docs.search_text("quarterly revenue", top_k=5)

# Override for one high-stakes query — probe harder, accept more latency.
docs.search_text("quarterly revenue", top_k=5, target_recall=0.99)
```

{% hint style="info" %}
`target_recall` is a *local* control. Over a server connection, `RemoteCollection.search` has no `target_recall` parameter, and the remote API does not expose `tune`/`apply_tuning` (though `build_ivf` is available remotely) — set the collection's default and tune it on the host, then serve it.
{% endhint %}

## Tuners and plans

Rather than hand-pick these values, attach a `Tuner`. It reads the collection (and an optional sample workload and objectives) and returns a `TuningPlan` — a full description of the recommended tradeoff, including a decision trace.

`Tuner` (`from telys import Tuner`) is the base contract: `tune_collection(collection, workload=None, objectives=None) -> TuningPlan`. It is abstract — the base method raises `NotImplementedError`. The concrete tuners ship outside the runtime-free SDK:

{% hint style="warning" %}
`HeuristicTuner` resolves lazily from the installed native runtime, and `AlgentaTuner` ships in the optional `telys-algenta` adapter. `from telys import HeuristicTuner` raises `ImportError` when no runtime is installed — neither concrete tuner is part of the pure-Python SDK. Install and verify the runtime first (`telys runtime install` → `telys runtime verify`).
{% endhint %}

The workflow is: produce a plan, inspect it, apply it, persist it.

```python
from telys import Telys, HeuristicTuner   # HeuristicTuner needs the installed runtime
from telys.embedding import AlgentaBigramEmbedder

db = Telys("./data")
emb = AlgentaBigramEmbedder()
docs = db.create_collection(
    "docs",
    dim=emb.profile.dimension,
    partition_by="scope",
    embedder=emb,
    tuner=HeuristicTuner(),
)

# 1. Produce a plan without changing anything.
plan = docs.tune(objectives={"target_recall": 0.95}, dry_run=True)

# 2. Inspect the tradeoff the plan proposes.
print(plan.target_recall)         # e.g. 0.95
print(plan.exact_crossover_rows)  # below this many rows, search stays exact
print(plan.ivf)                   # {"enabled": ..., "min_rows": ..., "target_recall": ...,
                                  #  "partitions": [{"key": ..., "rows": ..., "nlist": ..., "nprobe": ...}]}
print(plan.expected)              # predicted recall / latency
print(plan.decision_trace)        # why the tuner chose these values

# 3. Apply and persist the decision with the collection.
docs.apply_tuning(plan)
docs.save()                       # stores default_target_recall + applied_tuner
```

`tune(workload=None, objectives=None, dry_run=None)` controls whether the plan is applied: `dry_run=True` never applies, `dry_run=False` always applies, and `dry_run=None` applies only when the tuner's mode is `"suggest_then_apply"`. `apply_tuning(plan)` moves the dials — it sets `default_target_recall` and the exact-crossover, builds or reuses the per-partition IVF, and sets `nprobe` — returning a stats dict describing the result.

### What a TuningPlan contains

A `TuningPlan` is inspectable and serialisable; there is no `TuningPlan.save`, because applied tuning is persisted through `Collection.save()`.

| Field                           | Meaning                                                                                                                                       |
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `collection`, `tuner`, `engine` | what was tuned, by which tuner, on which engine                                                                                               |
| `target_recall`                 | the recall the plan aims for                                                                                                                  |
| `exact_crossover_rows`          | row count below which search stays exact                                                                                                      |
| `ivf`                           | IVF config: top-level `enabled`, `min_rows`, `target_recall`, and a `partitions` list where each entry carries per-partition `nlist`/`nprobe` |
| `expected`                      | predicted recall / latency for the plan                                                                                                       |
| `constraints`                   | limits the tuner respected                                                                                                                    |
| `decision_trace`                | ordered reasoning behind the choice                                                                                                           |
| `workload_hash`, `plan_id`      | provenance (`plan_id` auto-derives as `tune_<hash>`)                                                                                          |

Use `plan.as_dict()` or `plan.to_json(indent=2)` to log or diff a plan before you trust it.

## Next steps

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Why tune</strong></td><td>When tuning pays off and when to leave it alone.</td><td><a href="https://docs.telys.ai/tuning/why-tune.md">Why tune</a></td></tr><tr><td><strong>Tuner &#x26; TuningPlan</strong></td><td>The full API for tuners and the plan they produce.</td><td><a href="https://docs.telys.ai/tuning/tuner-and-plan.md">Tuner &amp; plans</a></td></tr><tr><td><strong>Run a tuning pass</strong></td><td>Produce a plan against a sample workload.</td><td><a href="https://docs.telys.ai/tuning/running-a-pass.md">Run a pass</a></td></tr><tr><td><strong>Search internals (IVF)</strong></td><td>How the index and probing actually work.</td><td><a href="https://docs.telys.ai/understand-telys/search-internals-ivf.md">Search internals</a></td></tr></tbody></table>
