Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Embedders

Reference for telys.embedding — the EmbeddingProvider base class, EmbeddingProfile, CallableEmbedder, and the Algenta bigram/multigram/pooled embedders.

Import embedders from telys.embedding; they are not top-level. An embedder turns text into (N, D) float32 vectors and carries an EmbeddingProfile that defines the vector space a collection lives in.

from telys.embedding import (
    EmbeddingProvider,
    EmbeddingProfile,
    CallableEmbedder,
    AlgentaBigramEmbedder,
    AlgentaMultigramEmbedder,
    AlgentaPooledEmbedder,
)

telys.embedding.__all__ is exactly those six names.

EmbeddingProvider (base class)

The base class every embedder subclasses. Implement these members in a subclass to define an embedder.

Member
Signature
Description

profile

-> EmbeddingProfile

The vector-space identity of this embedder.

embed_documents(texts)

list[str] -> [N, D] float32

Embed stored documents.

embed_queries(texts)

list[str] -> [N, D] float32

Embed queries. Defaults to embed_documents.

import numpy as np
from telys.embedding import EmbeddingProvider, EmbeddingProfile

class MyEmbedder(EmbeddingProvider):
    profile = EmbeddingProfile(provider="me", model_id="demo", model_version="1", dimension=8)
    def embed_documents(self, texts):
        return np.zeros((len(texts), 8), dtype="float32")
    # embed_queries falls back to embed_documents

assert isinstance(MyEmbedder(), EmbeddingProvider)  # plain base class — isinstance works by ordinary subclassing

EmbeddingProfile

A frozen dataclass that identifies a vector space. Two embedders are interchangeable only if their profiles produce the same space_id().

Field
Default
Meaning

provider

(required)

Vendor / family, e.g. "algenta", "openai".

model_id

(required)

Model name, e.g. "multigram", "text-embedding-3-small".

model_version

(required)

Version string of the model.

dimension

(required)

Vector dimension D.

dtype

"float32"

Element type.

normalization

"l2"

Vector normalization.

distance

"ip"

Distance metric (inner product).

pooling

"none"

Pooling strategy.

tokenizer_hash

""

Optional tokenizer identity.

projection_version

""

Optional projection identity.

created

""

Optional creation marker.

Methods:

Method
Returns
Description

space_id()

str

sha256: + 16 hex chars over the profile identity.

compatible_with(other)

bool

True when space_id() values match.

as_dict()

dict

Serializable form (persisted by Collection.save()).

CallableEmbedder

Wrap any function list[str] -> [N, D] float32 as an embedder — the bring-your-own path for OpenAI, BGE, sentence-transformers, or a bespoke function. Dense or lexical, whatever your function returns.

Parameter
Description

fn

list[str] -> [N, D] float32. Used for documents (and queries unless query_fn is given).

profile

The EmbeddingProfile for the space fn produces.

query_fn

Optional separate function for queries.

Each call asserts the output has ndim == 2 and shape[1] == profile.dimension.

Algenta embedders

The Algenta embedders are lexical hashers implemented in the native kernel. They take no space_id/model_id argument — space identity is computed from the profile.

AlgentaBigramEmbedder

Lexical bigram-hash embedder. Profile: provider="algenta", model_id="bigram", normalization="l2", distance="ip", pooling="bigram-hash". Dimension is read from the kernel (server hint: 64-d).

AlgentaMultigramEmbedder

Lexical embedder that concatenates [unigram | bigram | trigram], each L2-normalized blockwise. Profile: model_id="multigram", pooling="multigram-hash". Dimension is read from the kernel (MCP default hint: 384-d). This is the default embedder for MCP and telys mem.

AlgentaPooledEmbedder

Not implemented in the SDK. It resolves lazily from the runtime and raises ImportError without the runtime installed. Not available in a runtime-free install.

Choosing among them

Embedder
Kind
Needs
Typical use

AlgentaMultigramEmbedder

lexical

native kernel

Default; strong NL→code and keyword recall on-device.

AlgentaBigramEmbedder

lexical

native kernel

Smaller, faster lexical space.

AlgentaPooledEmbedder

(runtime-only)

full runtime

Advanced; not in the thin SDK.

CallableEmbedder

dense or lexical

your model

Bring OpenAI/BGE or any function.

See also

Last updated

Was this helpful?