Python API¶
The public API is importable from tabctx. Generated from the docstrings.
Estimators¶
TabctxClassifier ¶
TabctxClassifier(model: str | None = None, client: TabctxClient | None = None, max_request_bytes: int | None = MAX_REQUEST_BYTES, ttl_s: float | None = None)
Bases: ClassifierMixin, _TabctxEstimator
Classification on a tabctx deployment. Labels may be of any one type (strings, ints, bools); predict returns them as given to fit, and predict_proba's columns follow classes_.
TabctxRegressor ¶
TabctxRegressor(model: str | None = None, client: TabctxClient | None = None, max_request_bytes: int | None = MAX_REQUEST_BYTES, ttl_s: float | None = None)
Client¶
TabctxClient ¶
TabctxClient(base_url: str | None = None, tenant_id: str | None = None, timeout_s: float = 120.0, max_retries: int = 3, retry_backoff_s: float = 0.25, session_header: str = DEFAULT_SESSION_ID_HEADER, api_key: str | None = None)
base_url and api_key default to TABCTX_BASE_URL and TABCTX_API_KEY. A gateway needs the key (and sets the tenant itself); a deployment reached directly needs neither.
max_retries applies only to 503 backpressure (safe to retry by construction); every other error propagates immediately.
session_header: the affinity header name. Must match the deployment's RAY_SERVE_SESSION_ID_HEADER_KEY (default "x-session-id"); a deployment that renames it would otherwise silently lose replica affinity.
fit ¶
fit(X: ArrayLike, y: ArrayLike, task: Task | None = None, dataset_id: str | None = None, model: str | None = None, ttl_s: float | None = None) -> str
Encode and cache a training context; returns its dataset_id.
Omit task to let the server infer it from y (non-integer numbers mean regression). Supply a stable dataset_id when deploying multi-replica: a server-generated id can't be used to route the fit itself, so the context would land on an arbitrary replica. ttl_s: have the server delete it after this many seconds (capped by its own retention limit).
fit_file ¶
fit_file(file: TableFile, target: str | None = None, dataset_id: str | None = None, model: str | None = None, task: Task | None = None, ttl_s: float | None = None) -> str
fit() from a table file sent with the request: a path (streamed
from disk) or the file's bytes. Columns are the features plus the
label column, target (default: the last). Returns the dataset_id.
ttl_s: as for fit().
predict ¶
predict(dataset_id: str, X_test: ArrayLike | None = None, return_proba: bool = False, test_upload_id: str | None = None, model: str | None = None, quantiles: Sequence[float] | None = None) -> PredictResult
model: needed through a gateway, which routes predicts by it; a deployment reached directly knows it from the fit. quantiles (regression): also return each row's predicted distribution at these levels in (0, 1), as PredictResult.quantiles.
predict_file ¶
predict_file(dataset_id: str, file: TableFile, return_proba: bool = False, model: str | None = None, quantiles: Sequence[float] | None = None) -> PredictResult
predict() on a table file sent with the request (path or bytes). When the training table was a file, columns are matched by name.
fit_predict ¶
fit_predict(X: ArrayLike, y: ArrayLike, X_test: ArrayLike, task: Task | None = None, return_proba: bool = False) -> PredictResult
One-shot fit+predict+evict via the legacy endpoint. Prefer fit()+predict() when the same training set is queried again.
upload_csv ¶
Upload a CSV for later fit/predict-by-reference; returns the
upload_id. data is the raw CSV as bytes, or an open binary
file object (streamed -- see upload_csv_file). dataset_id is
required because it is the affinity routing key: the upload must
land on the same replica the fit/predict for that dataset will
reach.
fit_uploaded ¶
fit_uploaded(train_upload_id: str, dataset_id: str, task: Task | None = None, target_column: str | None = None, model: str | None = None, ttl_s: float | None = None) -> str
Fit-by-reference: consume a prior upload_csv() of a CSV whose columns are the features plus one target column (target_column, default: the last column). ttl_s: as for fit().
datasets ¶
Your tenant's contexts on the replica that answers, resident or spilled: id, model, task, shape, size, location and fit time. With several replicas this is only that replica's share.
delete ¶
Delete a fitted context from the deployment: device memory, host memory and its disk tier. Raises DatasetNotFoundError if there is none (never fit, already deleted, evicted, or lost to a restart).
tabular_models ¶
The ids of the tabular models behind this URL, in the order GET /v1/models lists them (see tabular_models_in).
default_model ¶
The model to use when none is named: the one tabctx's /v1/models marks as default; through a gateway that marks none, the first of KNOWN_MODELS it lists.
limits ¶
Capability discovery: what shapes will this deployment admit? Use it to validate a table client-side before uploading it.
PredictResult
dataclass
¶
PredictResult(predictions: list, probabilities: list[list[float]] | None, classes: list[str] | None, latency_ms: float, served_by: str | None, quantiles: list[list[float]] | None = None, quantile_levels: list[float] | None = None, cache_outcome: Literal['hit', 'restored'] | None = None)
Engine¶
TabctxEngine ¶
TabctxEngine(backend: TabularICLBackend | None = None, cache: ContextCacheManager | None = None, estimator: MemoryEstimator | None = None, backends: dict[str, TabularICLBackend] | None = None, estimators: dict[str, MemoryEstimator] | None = None, default_backend: str | None = None, host_guard: HostMemoryGuard | None = None, device_baseline_bytes: int = 0, device_available: Callable[[], int | None] | None = None, expiry_sweep_s: float | None = None)
Orchestrates one or more backends over a single shared context cache and GPU budget. Multi-backend serving (e.g. TabICL and TabPFN behind one endpoint) works because every cached context remembers which backend fit it (CachedContext.backend_name): fit() picks a backend, predict() dispatches to whichever one owns the context. Memory estimates are per-backend (models peak differently for the same shape) while admission headroom is global (they share the device).
Single-backend form: TabctxEngine(backend=, cache=, estimator=). Multi-backend form: TabctxEngine(backends={name: b}, cache=, estimators={name: e}, default_backend=name).
fit ¶
fit(X: ArrayLike, y: ArrayLike, task: Task | None = None, dataset_id: str | None = None, backend: str | None = None, schema: TableSchema | None = None, expires_at: float | None = None) -> str
Encode and cache a training context. Returns a dataset_id to pass to predict() -- generated automatically unless the caller supplies one (e.g. to reuse a stable, caller-known identifier).
expires_at (unix time): when the context is deleted, from every tier, whatever else happens (a retention limit); None keeps it until evicted or deleted.
schema (optional) records how the training table's columns were encoded (tabctx.schema) on the cached context, for callers that encode test tables the same way; the engine itself only ever checks feature count.
Raises InvalidInputError for malformed input (mismatched X/y lengths, empty tables, ragged rows) before anything else -- checked first so a careless client's bad request can't reach the backend as an unhandled exception.
Raises AdmissionRejected before any backend/GPU work if the training shape alone is estimated to exceed the configured memory ceiling (using the conservative formula-based estimator -- this pre-fit gate must stay conservative since nothing has run yet to measure).
The context's cache-accounting size (used for capacity/eviction bookkeeping, not for this admission gate) is queried from the backend AFTER fit() completes, not before: a real backend can measure actual post-fit GPU memory now that the fit has happened, which is far more accurate than any pre-fit estimate. Confirmed empirically this mattered -- the formula-based estimate ran ~14x higher than real measured GPU memory for a realistic multi-tenant shape, needlessly throttling how many contexts actually fit.
predict ¶
predict(dataset_id: str, X_test: ArrayLike, return_proba: bool = False, quantiles: Sequence[float] | None = None) -> PredictOutcome
Predict against a previously-fit context, reusing its cached training-context encoding. quantiles (regression only): also return each row's predicted distribution at these levels in (0, 1) (tabctx.uncertainty); InvalidInputError for a classification context. Large test sets are automatically chunked (see chunking.py) against the memory ceiling -- this is the one thing that must never let a single request take down the replica, which the naive one-shot wrapper this library replaces did not do.
A light predict (a resident context, at most LIGHT_PREDICT_ROWS test rows) runs beside a fit on this device; larger ones wait for the device's fits, and a spilled context is restored first.
fit_predict ¶
fit_predict(X: ArrayLike, y: ArrayLike, X_test: ArrayLike, task: Task | None = None, return_proba: bool = False) -> PredictOutcome
Convenience matching the one-shot semantics of the wrapper this library replaces: fit, predict once, evict. Prefer fit()+predict() directly when the same training set will be queried more than once -- that's the entire point of tabctx's caching.
evict ¶
Remove a context from every tier (device, host memory, disk); True if there was one. Its memory is freed now, or -- while a fit measures itself, or a predict still uses it -- once that is done.
contexts ¶
contexts() -> list[ContextInfo]
Every context this engine holds, resident or spilled (metadata only; never waits on GPU work).
schema ¶
The schema recorded at fit() for a cached context (None if it was fit without one). Reloads a spilled context as a side effect, exactly as predict() would; raises DatasetNotFoundError, as predict() would, when there is no such context.
ContextCacheManager ¶
ContextCacheManager(capacity_bytes: int, policy: EvictionPolicy | None = None, spill_store: Any | None = None, max_age_s: float | None = None)
spill_store: optional disk tier (cache/spill.py). When set, capacity-pressure evictions spill instead of dropping, and get() transparently reloads spilled contexts. Explicit evict() (caller intent, re-fit overwrite) never spills -- and also drops any spilled copy, so an evicted context can't resurrect.
put ¶
Insert a context, evicting via the policy to make room first. Returns the list of evicted dataset_ids. Raises CacheCapacityError if context.est_bytes alone exceeds total capacity -- no amount of eviction would ever make room for it. Overwriting an existing dataset_id replaces it in both tiers (a stale spilled copy must never outlive the context that superseded it); the id stays visible to has() throughout.
get ¶
The resident context, else a spilled one reloaded and re-admitted (its displacements may spill in turn). The reload runs outside the lock; callers serialize restores of one device (the engine does so under its fit lock).
delete ¶
Drop a context from every tier. Returns the resident one (the caller decides when its memory may be released) and whether there was a copy in any tier. Never spills: this is caller intent (an explicit delete, fit_predict's one-shot cleanup), and a spilled copy left behind would silently resurrect on the next get().
Atomic with respect to moves between tiers: it first waits for a spill, restore or replacement of this id to land (else a spill write finishing afterwards would bring the context back), then removes both copies under the lock, so no get() or evict_one() can see one copy gone and the other still there. The spilled copy goes first: if unlinking it fails, the resident copy is untouched. Its two unlinks are short, though a multi-GB file can take ~0.1 s, and bookkeeping (light predicts' peek and touch) waits meanwhile. Raises CacheUnavailableError if the move doesn't land in settle_timeout_s (nothing is deleted then: retry).
contexts ¶
contexts() -> list[ContextInfo]
Every context here, resident and spilled: metadata only. A context moving between tiers right now may be missing.
Values¶
PredictOutcome
dataclass
¶
PredictOutcome(predictions: list, probabilities: list[list[float]] | None = None, classes: list[str] | None = None, quantiles: list[list[float]] | None = None)
Result of a single predict() call.
predictions/probabilities/classes come from ONE inference pass -- a backend must never call predict() and predict_proba() separately for the same request (see backends/tabicl.py for why this matters).
concat
staticmethod
¶
concat(outcomes: Sequence[PredictOutcome]) -> PredictOutcome
Outcomes for consecutive row ranges, as one.
ContextInfo
dataclass
¶
ContextInfo(dataset_id: str, model: str, task: Task, n_train: int, n_features: int, bytes: int, location: Location, fitted_at: float | None, expires_at: float | None = None, device: str | None = None)
What a listing shows about one cached context: its metadata, never its contents.
EngineStats
dataclass
¶
Errors¶
tabctx error types.
Kept HTTP-agnostic on purpose -- a serving layer (e.g. serve/app.py) maps these to status codes at its own boundary, so the engine stays usable outside of any particular transport.
TabctxError ¶
Bases: Exception
Base class for all tabctx errors.
InvalidInputError ¶
Bases: TabctxError
Raised for malformed input (mismatched X/y lengths, empty tables, ragged rows, a predict() feature count that doesn't match the cached context) -- checked before any backend/GPU work is attempted. Found via load testing: without this, malformed input reached the backend and raised an unhandled, untranslated exception (numpy/sklearn shape errors), surfacing as a raw 500 that leaked no useful detail to the caller -- exactly the failure mode a multi-tenant service can't afford (one careless client's bad request shouldn't look like a server bug).
AdmissionRejected ¶
Bases: TabctxError
Raised when a request's estimated memory footprint exceeds the configured ceiling, before any backend/GPU work is attempted.
DatasetNotFoundError ¶
Bases: TabctxError
Raised when predict() is called with a dataset_id that isn't cached (never fit, evicted, or lost to a replica restart).
CacheCapacityError ¶
Bases: TabctxError
Raised when a single context's estimated size alone exceeds the cache's total capacity -- no amount of eviction would make room.
UploadNotFoundError ¶
Bases: TabctxError
Raised when fit()/predict() references an upload_id that isn't present on this replica (never uploaded, expired, already consumed, or -- in a multi-replica deployment -- uploaded without the session affinity header, so it landed on a different replica).
UploadTooLargeError ¶
Bases: TabctxError
Raised when a streamed upload exceeds the configured size cap, before the excess is written to disk.
BackendComputeError ¶
Bases: TabctxError
Wraps a backend-level compute failure (e.g. a CUDA OOM that slipped past admission control) so callers don't need to import torch to catch tabctx errors.
CacheUnavailableError ¶
Bases: TabctxError
Raised when contexts can't be listed or deleted correctly right now: a GPU worker is dead (its spilled copies would outlive a delete) or didn't answer, or a context is still moving between cache tiers. Nothing is deleted where it couldn't be done; retryable, so the API answers 503.
HostMemoryUnavailableError ¶
Bases: TabctxError
Raised when host-memory admission can't be decided right now (its shared accounting is stuck, e.g. a worker died holding the lock); retryable, so the API answers 503.