> For the complete documentation index, see [llms.txt](https://docs.telys.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.telys.ai/understand-telys/tuning-tradeoffs.md). # Tuning & tradeoffs Search quality is a negotiation between three quantities that pull against each other: **recall** (did you find the true nearest neighbours?), **latency** (how long the query took), and **memory** (index overhead). Telys exposes the tradeoff through a small surface — an exact-vs-approximate crossover, IVF knobs, and a default recall target — and lets a `Tuner` choose good values and hand back a `TuningPlan` you inspect before applying. ## Three quantities in tension You cannot maximise all three at once; tuning is about picking where you sit. | Quantity | You raise it by… | And you pay in… | | --------------- | -------------------------------------------------------- | ------------------------------------ | | Recall | a higher `target_recall`, more IVF probes | latency | | Latency (lower) | fewer probes, smaller `top_k`, staying exact while small | recall at scale | | Memory | building an IVF index for speed | index overhead on top of the vectors | The right point depends on your workload. An agent's long-term memory that fits in tens of thousands of rows wants exact, high-recall search; a multi-million-row corpus wants an IVF index tuned for latency at an acceptable recall. ## Exact vs approximate: the crossover Below a row threshold — the **exact-crossover** — Telys answers with an exact scan of the partition: 100% recall, and fast because the set is small. Above it, Telys switches to the **IVF** (inverted-file) index: vectors are grouped into `nlist` cells and each query probes the `nprobe` closest cells instead of scanning everything, trading a sliver of recall for a large latency drop at scale. `build_ivf(min_rows=20000, target_recall=0.98)` builds that index explicitly. The default `min_rows=20000` is the point past which an index earns its keep; `target_recall` steers how many cells a query must probe to hit the desired recall. ```python from telys import Telys from telys.embedding import AlgentaBigramEmbedder db = Telys("./data") docs = db.open_collection("docs", embedder=AlgentaBigramEmbedder()) # Build a per-partition IVF once the collection is large enough to benefit. docs.build_ivf(min_rows=20000, target_recall=0.98) docs.save() ``` ## The recall dial A collection carries a `default_target_recall`. Every `search`/`search_text` uses it unless a call overrides it — set the baseline once and raise the bar only for queries that need it: ```python # Uses the collection's default_target_recall. docs.search_text("quarterly revenue", top_k=5) # Override for one high-stakes query — probe harder, accept more latency. docs.search_text("quarterly revenue", top_k=5, target_recall=0.99) ``` {% hint style="info" %} `target_recall` is a *local* control. Over a server connection, `RemoteCollection.search` has no `target_recall` parameter and the remote API does not expose `tune`/`apply_tuning`/`build_ivf` — set the collection's default and build its index on the host, then serve it. {% endhint %} ## Tuners and plans Rather than hand-pick these values, attach a `Tuner`. It reads the collection (and an optional sample workload and objectives) and returns a `TuningPlan` — a full description of the recommended tradeoff, including a decision trace. `Tuner` (`from telys import Tuner`) is the base contract: `tune_collection(collection, workload=None, objectives=None) -> TuningPlan`. It is abstract — the base method raises `NotImplementedError`. The concrete tuners ship outside the runtime-free SDK: {% hint style="warning" %} `HeuristicTuner` resolves lazily from the installed native runtime, and `AlgentaTuner` ships in the optional `telys-algenta` adapter. `from telys import HeuristicTuner` raises `ImportError` when no runtime is installed — neither concrete tuner is part of the pure-Python SDK. Install and verify the runtime first (`telys runtime install` → `telys runtime verify`). {% endhint %} The workflow is: produce a plan, inspect it, apply it, persist it. ```python from telys import Telys, HeuristicTuner # HeuristicTuner needs the installed runtime from telys.embedding import AlgentaBigramEmbedder db = Telys("./data") emb = AlgentaBigramEmbedder() docs = db.create_collection( "docs", dim=emb.profile.dimension, partition_by="scope", embedder=emb, tuner=HeuristicTuner(), ) # 1. Produce a plan without changing anything. plan = docs.tune(objectives={"target_recall": 0.95}, dry_run=True) # 2. Inspect the tradeoff the plan proposes. print(plan.target_recall) # e.g. 0.95 print(plan.exact_crossover_rows) # below this many rows, search stays exact print(plan.ivf) # {"enabled": ..., "min_rows": ..., "target_recall": ..., # "partitions": [{"key": ..., "rows": ..., "nlist": ..., "nprobe": ...}]} print(plan.expected) # predicted recall / latency print(plan.decision_trace) # why the tuner chose these values # 3. Apply and persist the decision with the collection. docs.apply_tuning(plan) docs.save() # stores default_target_recall + applied_tuner ``` `tune(workload=None, objectives=None, dry_run=None)` controls whether the plan is applied: `dry_run=True` never applies, `dry_run=False` always applies, and `dry_run=None` applies only when the tuner's mode is `"suggest_then_apply"`. `apply_tuning(plan)` moves the dials — it sets `default_target_recall` and the exact-crossover, builds or reuses the per-partition IVF, and sets `nprobe` — returning a stats dict describing the result. ### What a TuningPlan contains A `TuningPlan` is inspectable and serialisable; there is no `TuningPlan.save`, because applied tuning is persisted through `Collection.save()`. | Field | Meaning | | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `collection`, `tuner`, `engine` | what was tuned, by which tuner, on which engine | | `target_recall` | the recall the plan aims for | | `exact_crossover_rows` | row count below which search stays exact | | `ivf` | IVF config: top-level `enabled`, `min_rows`, `target_recall`, and a `partitions` list where each entry carries per-partition `nlist`/`nprobe` | | `expected` | predicted recall / latency for the plan | | `constraints` | limits the tuner respected | | `decision_trace` | ordered reasoning behind the choice | | `workload_hash`, `plan_id` | provenance (`plan_id` auto-derives as `tune_`) | Use `plan.as_dict()` or `plan.to_json(indent=2)` to log or diff a plan before you trust it. ## Next steps
Why tuneWhen tuning pays off and when to leave it alone./pages/dUb8SpYWDKmPq4apsKnQ
Tuner & TuningPlanThe full API for tuners and the plan they produce./pages/NznoaOZPDT34Q6o79zU5
Run a tuning passProduce a plan against a sample workload./pages/oRgOGxUrNbTlcSvpJeau
Search internals (IVF)How the index and probing actually work./pages/34ekyTEaG3dG24136d0C
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.telys.ai/understand-telys/tuning-tradeoffs.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.