Skip to content

Changelog

[Unreleased]

Fixed

  • A predict reported cache_outcome: hit when the table was restored from disk while its rows were being read, as on a replica after a restart (its schema not yet recorded); the restore is now counted wherever it happens.

Changed

  • cache_outcome is required in the predict response's schema (the server always sent it; generated clients now see it as always present). The coalescing batcher closes a batch and releases everyone in it even if its window's wait fails.

[0.19.0] - 2026-11-05

Fixed

  • A 422 for a non-finite number in a request (Infinity) no longer becomes a 500: the error echoes the value as text.
  • /readyz no longer waits behind long fits (#51). It read GPU memory through torch's allocator (which waits for a fit freeing memory), asked each GPU worker for a report (up to 30 s behind its queued calls), took the host-memory guard's cross-process lock and walked the spill directory; on prod, readiness timed out during large fits, so a busy replica looked hung, and with the Helm chart's single replica that would take it out of its Service. Now every one of those is measured on its own thread every 5 s (worker processes send theirs), and /readyz reads the latest results and the guard's counters without a lock. It reports how old the device measurements are (device_report_age_s), and fails only for a dead worker or one silent for 10 minutes. Polled every 0.2 s during a 49 s TabICL fit on a CPU: median 1 ms, at most 353 ms. /v1/tabctx/limits reads the cache's counters without its lock too. A worker that never reports goes stale from its start; a spill tier that can't be read is shown as an error rather than a stuck worker. Not yet verified on a GPU: on a single-GPU replica serving in-process, whether torch releases the interpreter lock while its measuring call waits on the allocator; replicas with GPU worker processes measure in those processes, out of /readyz's way.

Added

  • Predict responses and PredictResult expose cache_outcome: hit for a resident context or restored after disk spillover. Coalesced requests share their batch's outcome; older deployments leave the client field as None. (#112; the project's first outside contribution, from @sunxiuguo in #115.)
  • Retention limits (#103). TABCTX_MAX_CONTEXT_AGE_S caps how long any fitted table is kept after its fit, including tables fit before it was set (one spilled by an older tabctx, with no recorded fit time, counts as expired); a fit can ask for less with ttl_s, counted from when the request arrives (HTTP; TabctxClient.fit, fit_file and fit_uploaded; the scikit-learn estimators, which apply it again on every refit). At that time the context is deleted from device memory, host memory, the disk tier and the server's schema cache, whether or not anyone uses it again: a predict after it is a 404, and each engine sweeps expired contexts every 30 s (in worker processes too), re-checking each under the cache lock so a refit of the same id is never deleted. Fit responses and GET /v1/tabctx/datasets show expires_at (the listing leaves out expired tables); /v1/tabctx/limits shows the server's limit; the Helm chart sets it with maxContextAgeS.
  • "Why a server", a docs page for people arriving from the models' own packages: what changes when a model serves many tables, users and programs (repeat predicts, many tables per GPU, out-of-memory, keeping a fit, uncertainty, licensing), what tabctx does at each step, and when not to use it. Each need links to a thread where users of these models raised it.
  • TabDPT, a third model (#102): TABCTX_BACKEND=tabdpt (or tabicl,tabdpt) serves Layer 6's TabDPT as tabdpt-1.3. Its code and weights are Apache-2.0, so with TabICLv2 there are now two models a business can serve without a separate license. Install with pip install "tabctx[tabdpt]"; the container image includes it.
  • On credit-g (the quickstart's split) it scored 0.768 against TabICL's 0.772; on California housing, R² 0.89, with its 90% interval holding 88.4% of held-out prices (python examples/prediction_intervals.py --model tabdpt-1.3), on a CPU. Classification, regression, prediction intervals, list and delete, and the spill tier all work as for the other models.
  • TabDPT keeps no encoding of the training table: its fit stores the table, and each predict samples a context of TABCTX_TABDPT_CONTEXT_SIZE rows (2,048) from it, with TABCTX_TABDPT_ENSEMBLES (4) ensemble members for both tasks. Left unbounded, as TabDPT's default is, a predict's cost grows with the square of the table (8,000 rows: 45 s on a CPU, against 4 s). tabctx adds serving: one endpoint, admission control, tenancy.
  • One network per device, shared by every context and both tasks; the spill tier stores a context without it, tagged with the checkpoint version.
  • Flash attention on every CUDA device (TabDPT enables it only for the device name "cuda", so a multi-GPU replica's "cuda:1" ran without it).
  • Predicts use a fixed seed, so the same request gives the same answer; the mean is the same with or without quantiles.
  • Numeric labels keep numeric order (1, 2, 10), as with the other models. Bad test rows (an infinite value) are a 422.
  • A Helm chart (#105) in deploy/helm/tabctx, with a "Deploy on Kubernetes" guide. One tabctx pod with its GPUs:
  • /livez (startup, liveness) and /readyz (readiness) probes, with the startup budget covering the first weights download and kernel compilation;
  • a persistent volume for weights and compiled kernels, and an opt-in one for the disk tier (both kept on uninstall, or bring your own claim);
  • a TabPFN token from an existing Secret;
  • the Pod Security "restricted" profile (non-root, no capabilities, RuntimeDefault seccomp);
  • a 300 s shutdown grace for in-flight fits;
  • the settings as values, plus extraEnv (which replaces a variable the chart sets rather than duplicating it) and extra volumes.
  • Rendering refuses what would fail at runtime: tabpfn without a token, replicaCount > 1 with ReadWriteOnce volumes, sizes under 1 GiB.
  • CI lints it and checks those refusals; the chart's version must match the package's (a unit test). Release tags push it to oci://ghcr.io/vectorinstitute/charts/tabctx.
  • Installed on a local kind cluster (CPU, fake model): fit, predict, list and the tenant check worked through the Service, and the non-root user could write its volumes.
  • A documentation site at https://vectorinstitute.github.io/tabctx/ (#9), built with MkDocs Material and published from main by .github/workflows/docs.yml. Pull requests build it with --strict, so broken links and missing API symbols fail. It has:
  • a landing page;
  • a quickstart, and guides for Python and scikit-learn and for the HTTP API;
  • the serving, agents, architecture and benchmark docs;
  • a reference generated from the code: the Python API from docstrings, the HTTP API from the service's own OpenAPI spec (rendered with Redoc), and configuration;
  • a guide to adding a model.
  • The OpenAPI spec names the service tabctx at its real version, groups routes (contexts, deployment, health, legacy) and gives each a summary.
  • Prediction intervals for regression. A predict can ask for quantiles (levels in (0, 1), e.g. [0.05, 0.5, 0.95] for the median and a 90% interval), and each test row gets the predicted distribution's value at each level. TabICL and TabPFN both predict a distribution over the target, so the quantiles come from the same forward pass as the mean: no extra GPU work. examples/prediction_intervals.py measures how well calibrated they are: on California housing (TabICL, 2,250 training and 750 held-out rows) the 90% interval held 86.5% of the prices, a little narrower than nominal.
  • HTTP: quantiles in POST /v1/tabctx/predict, returned as quantiles and quantile_levels. JSON takes a list; a multipart form takes one field, comma- or space-separated or a JSON array (a repeated field is refused, as for every form field). Bad levels are a 422, as are quantiles for a classification context (it returns probabilities) or from a model that predicts only a point. The legacy /v1/tabicl/predict doesn't take them.
  • Python: TabctxRegressor.predict_quantiles(X, quantiles=(0.05, 0.5, 0.95)) returns an (n_samples, n_levels) array (bad levels raise ValueError). TabctxClient.predict(..., quantiles=...) and predict_file return PredictResult.quantiles and quantile_levels, check the levels before sending, and raise if the deployment predates quantiles instead of silently returning none.
  • Engine and backends: quantiles= on predict throughout; the backend protocol documents it, and a backend without it still serves every other predict. Concurrent predicts are only coalesced with others asking for the same levels.
  • List and delete contexts (#103, part 1). GET /v1/tabctx/datasets lists the caller's contexts, resident or spilled: id, model, task, shape, size, location, GPU and fit time (metadata only). DELETE /v1/tabctx/datasets/{dataset_id} removes one from device memory, host memory and the disk tier on every GPU, or answers 404 if there is none. Both are scoped by tenant like every /v1/tabctx route. See "Listing and deleting contexts" in docs/serving.md.
  • A delete is atomic with respect to spills and restores: one in progress lands first, so no copy reappears afterwards. If it doesn't land in time, the delete answers 503 having deleted nothing.
  • On a multi-GPU replica the delete asks every GPU at once, not just the ones the residency snapshot lists. It never reports success while a copy may survive: while a GPU worker is down (its spilled copies would come back with it), list and delete answer 503 before touching anything, and a GPU failing mid-delete makes it 503 too. The client retries, and a retry that finds the context gone counts as deleted.
  • Worker processes answer listings and deletes from a small pool of their own, so they wait neither behind queued fits nor in front of /readyz.
  • Client: TabctxClient.datasets() and TabctxClient.delete(dataset_id).
  • Agents: a delete_model tool (and tabctx-mcp delete <name>) removes a model's context on the server and its saved files, which include its training table. A context another saved model shares (same table, model and task) is kept, and a model whose signature doesn't verify still loses its files without the server being asked. Saved models now record their context id.
  • Spilled contexts now record their fit time; ones spilled by older versions list it as null.
  • A new error, CacheUnavailableError (503), for "can't list or delete correctly right now".

Fixed

  • A caller without a tenant id could reach a tenant's context by naming its scoped id (t:<tenant>:<id>) on deployments that don't require tenants: predict on it, overwrite it, and now delete it. Ids starting with t: are now refused (422) for callers without a tenant id.
  • TabctxClient's GET requests send the tenant header, as its POSTs always did, and requests without a body no longer send a Content-Type.
  • TabctxClient raised UploadNotFoundError for any 404 whose message contained "upload", including a missing dataset with "upload" in its id. It now matches only the upload store's own message.

Changed

  • The unit tests run in parallel (pytest -n auto, with pytest-xdist in the dev dependencies), as CI now does.

[0.18.0] - 2026-11-04

Changed

  • A shorter README with a logo and a headline figure. The front page now has the pitch, a fit-vs-predict chart from a saved A40 sweep (benchmarks/plot_readme_figure.py draws it from benchmarks/baselines/sweep-a40x4-tabicl-v2.json), a quickstart that runs on a laptop with no API key (checked against a fresh install of 0.17.1 from PyPI), the agent tools, a comparison with in-process models and hosted APIs, and each model's weights license. The detail moved, unchanged, into docs/serving.md, docs/how-it-works.md, docs/benchmarks.md and docs/coding-agents.md. Images use absolute URLs so the PyPI page shows them too.

Fixed

  • The estimators and tabctx-mcp use the deployment's own default model when none is named, instead of always tabpfn-3.5. Before, TabctxClassifier(), TabctxRegressor() and the agent tools' fit_model/explain failed with a 422 on any deployment that doesn't serve TabPFN, and evaluate reported an error for TabPFN there.
  • model=None (the default) is resolved at fit time from GET /v1/models, and the estimators record the answer as model_. The context id is computed from that resolved model, so naming the default explicitly reuses the same cached context.
  • tabctx's own listing marks its default. A gateway lists its whole catalog with no mark: Vector's lists 65 LLMs beside tabicl-v2 and tabpfn-3.5. There, the client narrows the list to the tabular models it knows (tabctx.client.KNOWN_MODELS) and prefers tabpfn-3.5, as before.
  • evaluate with no models runs every tabular model served, and never a gateway's LLMs.
  • New: TabctxClient.tabular_models() and TabctxClient.default_model().
  • GET requests now retry on 503, like fit and predict.
  • The agent tools check their arguments and read the table before asking the server.
  • pip install "tabctx[client]" installs pandas. The estimators are built for DataFrames, and scikit-learn's loaders (fetch_openml(as_frame=True)) need pandas, so the documented quickstart failed on a clean install.
  • from tabctx import TabctxClient works (also PredictResult). Before, the client could only be imported from tabctx.client.

Removed

  • tabctx.estimators.DEFAULT_MODEL and tabctx.mcp_tools.tools.MODELS: the default now comes from the deployment. tabctx.client.KNOWN_MODELS lists the tabular model ids the client recognises in a gateway's catalog.

[0.17.1] - 2026-11-01

Changed

  • pi in the agent demo now runs on Vector's on-prem Qwen3.8 27B (Qwen3_8-27B on the gateway) instead of Claude Sonnet 5.5, so one of the three agents uses an open model on Vector's own GPUs. A hosted visitor whose project lacks it still gets Sonnet. PI_MODEL overrides it as before.

Fixed

  • Demo page: an agent's progress notes render its bold and code instead of showing raw ** and backticks (Qwen uses them a lot). Only those inline tags are allowed through.

[0.17.0] - 2026-10-31

Added

  • A Ray-free standalone host: python -m tabctx.serve (the new standalone extra: [serve] minus ray) serves the same routes as the Ray Serve deployment from one plain uvicorn process, for platforms that don't run Ray -- e.g. a Kubernetes Deployment behind a gateway. The service moved to serve/service.py, host-agnostic and importable without ray (route handlers unchanged; ROUTE_SPECS captures the routes at import time so serve/standalone.py's make_app() can rebind them to a plain instance); serve/app.py keeps its import path and is now only the Ray host (the deployment settings and the consistent-hash affinity router). Serve-isms get standalone replacements: GET /livez answers 503 exactly when check_health would fail the replica (a dead per-GPU worker -- /readyz is the wrong liveness probe, it also 503s on transient pressure), and a request limiter replicates max_ongoing_requests/max_queued_requests shedding (TABCTX_MAX_ONGOING_REQUESTS/TABCTX_MAX_QUEUED_REQUESTS, defaults 8/16, probes exempt). One replica only: nothing outside Ray routes a dataset's predicts back to the replica that fit it. Unit-tested over httpx's ASGI transport in the ray-less unit environment (fit/predict inline and multipart, tenancy, 404/422, limiter, livez); the integration suite still proves serve/app.py on a live Ray cluster.
  • A serving image (Dockerfile, pushed to ghcr.io/vectorinstitute/tabctx:<version> by .github/workflows/docker_image.yml on each release tag): python:3.12-slim + a pinned torch (PyPI wheels bundle the CUDA runtime; the host supplies only the driver) + tabctx[standalone,tabicl,tabpfn], entrypoint python -m tabctx.serve --host 0.0.0.0, non-root. Weights are mounted, not baked. CI smoke-tests the image against the fake backend (imports, readyz/livez, tenant-gated fit and predict); GPU validation stays with the deploying platform's rehearsal. Every dependency except torch's own stack is installed at its uv.lock version (CI checks it), so a PR's smoke build and the release ship the same model libraries; the image carries gcc so torch.compile can build Triton's C helper instead of silently running eager kernels (about 1.3x slower).

Changed

  • The agent demo is now about hospital readmission, not telecom churn. It uses the public Diabetes 130-US Hospitals table (UCI; OpenML #4541): 30,000 past stays and 2,000 current patients who appear nowhere in the history, picked in proportion to their number of stays so both sides have the same mix. The agent ranks the current patients by their risk of coming back within 30 days, finds the drivers, and tests care changes with what-if predictions. Admission and discharge codes are named, and diagnosis codes grouped, so tables and what-if rows read as "Home health service", not "6". Claude Code, Codex and pi all reached ROC AUC 0.674 on the real outcomes, with 8 of their top 10 and 36% of their top 100 readmitted (base rate 11.0%). examples/agent_demo/task.py now holds the files, columns, outcome labels and task steps, which the runner, scoring and page read.
  • Demo page: less text. The key note became a tooltip, agent cards show icon chips instead of two lines of text, the task shows as five step icons with the prompt folded, a failed tool call shows one line, and the hero lost its paragraph and footer. The predict card shows the expected split (mean probability), which says something even when almost no row crosses 50%.

Fixed

  • /readyz no longer fails on a healthy, busy replica. A worker answered /readyz's report from the same 16 threads that run fits and predicts, and fits on one device run one at a time, so with more than 16 calls queued on a device the report waited past its 30 s timeout and readiness returned 503. The report now has its own thread in each worker.
  • examples/agent_demo/run.sh claude failed on macOS's bash 3.2 without CLAUDE_MODEL set (an empty array is "unbound" under set -u).
  • make_showcase.py masks the login name, which ls -l output in a transcript shows.
  • Demo page tooltips escape untrusted values (column names, class labels, what-if values, tool errors) a second time, since the tooltip renders its attribute as HTML.

[0.16.0] - 2026-10-31

Added

  • tabctx-mcp: tabular foundation models as tools for coding agents (tabctx/mcp_tools/, pip install "tabctx[mcp]"). There are seven tools: inspect_table, evaluate, fit_model, predict, what_if, explain and list_models. They come in four forms, all over one implementation (tools.TabularTools):
  • an MCP server (tabctx-mcp, stdio), for Claude Code, Codex and Gemini CLI;
  • shell commands that print JSON (tabctx-mcp fit ...);
  • tabctx-mcp tools and tabctx-mcp call, which expose and call the MCP tools without MCP;
  • a pi extension (tabctx-mcp pi-extension) that registers them as native pi tools, with no shell involved.

How the tools behave: - Fitted models are saved by name in .tabctx-mcp/ (or $TABCTX_MCP_HOME), and fit the server again if it dropped their context. - evaluate scores both models against scikit-learn gradient boosting on held-out rows. - predict writes the table with prediction and probability columns, and returns the top-ranked rows (rank_by, where, show), with the id column found automatically. - what_if can start from a real row (base_file, base_where) and change only some columns. - explain is permutation importance, with all shuffled copies sent in one predict. - Tables are CSV, TSV or Parquet. Blank cells are missing, and numbers stored as text become numbers. - Errors reach the agent as MCP tool errors with their message: bad arguments, the deployment's 4xx answers, and file or network failures. - tabctx-mcp skill prints a guide for agents (SKILL.md).

Further behaviour: - Saved models are signed with HMAC under a per-user key (~/.config/tabctx-mcp/signing.key, mode 0600). A model planted in a project's .tabctx-mcp/ is neither listed nor unpickled. - evaluate keeps the other models' results when one fails. - predict writes only inside the working directory, and never over its input. - Class labels stay "0"/"1" when blank target cells make pandas read them as floats. - Zero-padded codes such as "02139" stay text, and where filters match numbers numerically. - Results are valid JSON: NaN becomes null. - explain caps its batched predict at 60,000 rows.

examples/agent_demo/ runs a churn task with Claude Code, Codex and pi through the prod gateway. All three reached ROC AUC 0.866-0.867 on the real outcomes, and 9 of each one's 10 top-ranked customers churned. - examples/agent_demo/ui/: a local web page that runs or replays an agent on the churn task. Each tabctx call appears as a chart card: column map, model comparison, predicted split and risk table, importance, what-if deltas. The scores against the real outcomes come last. The UI is FastAPI with Server-Sent Events and plain HTML, CSS and JS. - CI's unit and integration jobs install the mcp extra (which pulls in client), so the estimator tests, which were skipped there before, now run. - benchmarks/bench_upload_formats.py: the same tables through a gateway as JSON and as Parquet, interleaved, splitting each fit's time between the hop to tabctx and tabctx's own work. On prod (0.15.0, 12 fits each of 5,000 x 500 and 10,000 x 100 TabICL): Parquet bodies were 8.5 MiB vs 14.0 MiB, the hop's median 1.2 s vs 2.7 s, its worst 42 s vs 113 s, and 1 fit in 12 took over 10 s vs 3 in 12; tabctx's own work was the same. Baselines prod-bench-0.15.0.json and upload-formats-0.15.0.json.

Fixed

  • A Parquet file's pandas index was read as data. DataFrame.to_parquet() saves any index except the default 0..n-1 range as extra columns: __index_level_0__ (after train_test_split, say), or the index's name. The server read those columns as features, so:
  • the model learned from row numbers;
  • predicting with JSON rows, or a file without the index, got a 422 ("the table has 20 columns; the training table had 21");
  • with no target given, the index, which comes last, was used as the labels.

Columns the file's pandas metadata lists as its index are now dropped when the file is read (serve/files.py), the way pandas reads them back as the index. One named as the target is kept. Found through the gateway, fitting a file uploaded with /v1/tabctx/upload.

Other changes and limits: - Behaviour change: a meaningful named index, such as customer_id or a date DatetimeIndex, used to be a feature and no longer is, matching pandas. To use it, save the frame after reset_index(), or name it as the target. - When the index removal leaves too few columns, the error now says the index was removed. - Pandas metadata that isn't in the shape pandas writes is ignored. - Still open: DataFrame.to_csv() writes the index as an unnamed first column. CSV files have no metadata to recognize it by, so write CSVs with index=False.

[0.15.0] - 2026-10-21

Added

  • scikit-learn estimators (#63): TabctxClassifier and TabctxRegressor (tabctx/estimators.py, from tabctx import ...; pip install "tabctx[client]" adds scikit-learn and pyarrow). They take DataFrames, numpy arrays or lists of rows with text columns, dates and missing values as they are; labels are checked as scikit-learn checks them and come back exactly as given (sent as class indices); predict_proba follows classes_; clone, pipelines and model selection work. Test columns match the training columns by name when both have names, else by position (with a warning); a column that was text at fit stays text at predict, whatever its new values look like. Each table is one Parquet file (tabctx/table_files.py, float32 numbers, timestamps and durations as such); predicts over max_request_bytes (default 29 MiB, under the Vector gateway's 30 MiB) are split until each part fits, and a training table over it is refused before sending. The context id comes from the table, labels, model and task; a predict that finds its context gone (the server's "no cached context") fits it again from the kept training file and retries once, with the model it was fit with.
  • Client: TabctxClient(api_key=...) (sent as Authorization: Bearer, for the Vector gateway); base_url and api_key default to TABCTX_BASE_URL and TABCTX_API_KEY. fit_file() / predict_file() send a table file (path, streamed, or bytes) in the request as multipart/form-data; predict(model=...) for gateways, which route predicts by model. Redirects are refused rather than followed (urllib would carry the key to the new host, and turn a POST into a GET). The key never appears in the client's repr and is dropped from pickles: an unpickled client (a saved model, a worker with n_jobs > 1) reads TABCTX_API_KEY, and a 401 without a key says so; copies (sklearn's clone) share the client. 403 maps to PermissionError like 401.
  • A table file in the fit or predict request itself. /v1/tabctx/fit and /v1/tabctx/predict take multipart/form-data as well as JSON: the same fields as form fields, the table as a file part (curl -F dataset_id=churn -F target=churned -F file=@train.parquet), so a file is one call instead of upload-then-reference. The file streams to the replica's disk under the upload size cap, is read, and is deleted. OpenAPI documents both body types.
  • Parquet and compressed CSV. Files (sent with the request or through /v1/tabctx/upload) may be CSV (plain, gzip or zstd) or Parquet, detected from the content (serve/files.py, pyarrow, replacing the numpy CSV parser). A 5,000 x 500 float table is 51 MB as JSON and 13 MB as Parquet; reading it takes 16 ms. Dates and times become seconds, stored from the training minimum so float32 keeps sub-second resolution (TableSchema.offsets, for any float64 column past 2^24); header names are stripped; decimal targets are numbers. Labels keep their text as written ("1.0", "01", "true" are classes of their own; "NA" can be a class among text labels); a missing value among numeric labels, in any spelling, is a 422 naming the row, and regression with text labels is a 422. Reading a CSV peaks at about 3x its size in host memory (arrow's parser; Parquet much less), which matters only for multi-GB direct uploads. pyarrow and python-multipart join the serve extra.
  • Multipart bodies stream to disk under the upload cap (serve/multipart.py): the file part goes straight to the upload writer as it arrives, so an oversized body stops at the cap (or is refused from its Content-Length before any is read), a good one is written once, and disk writes run off the event loop. Tenant checks run before the body is read.
  • Text columns (#63). Feature cells may be text in JSON rows and files alike. At fit, each text column's values are coded in sorted order (tabctx/schema.py, TableSchema); the encoding is kept with the context (spill included) and applied at predict, where a value the training table never had becomes missing. Text where the training table had numbers is a 422 naming the column. All-number JSON tables still convert in one numpy call (validation takes ~20% longer for the wider cell type: 88 -> 108 ms for a 5,000 x 500 body).

Changed

  • Test files are matched to the training file's columns by name: any order, and extra columns (such as the target) are ignored; a missing column is a 422 naming it. A reordered header used to be a 422.
  • A request with more than one table (inline rows, a file, an upload id) is a 422 checked before an upload is used up, so the upload can still be used by a corrected request.
  • Each replica keeps the schemas of the contexts it fit (serve/schemas.py, LRU bounded in entries and bytes; fits of one id record in turn, and a failed fit leaves the engine's copy to answer), so a predict doesn't ask a worker process for it; after a restart it falls back to the engine's copy. engine.schema() raises DatasetNotFoundError for an unknown context (a predict against one is a 404 whatever its rows are).
  • CachedContext.feature_names / engine.feature_names() became CachedContext.schema / engine.schema() (a TableSchema: names and text categories); fit(feature_names=...) is fit(schema=...). Spill files from earlier versions load with their column names, and new ones still record feature_names for a rollback.

[0.14.1] - 2026-10-21

Fixed

  • TabICL compiled new graphs inside requests (#47): call shapes warmup didn't cover -- tables of ~20k+ rows (differently strided inputs), 1,024+-row regression predicts, and variants that overlapping calls on one worker produce -- each compiled 1-7 graphs inside the request that hit them, 5-25 s on an A100 and ~4x that on prod's A40s, holding the device so other models' calls waited too. On prod, a 5,000 x 500 TabICL fit compiled 3 graphs on every call (25.8 s, timings_ms from 0.14.0). Once every backend in a process has warmed up (serve/factory.finish_warmup: after all devices of an in-process engine, or a worker process's one), a call no compiled graph fits now runs eagerly instead (torch.compiler.set_stance("eager_on_recompile"), process-wide, only where compiled blocks were installed). On an A100: a first 20,000 x 100 fit 12.5 s -> 3.0 s, a first 1,000-row regression predict 1.8 s -> 0.14 s; eager costs ~30% on those families once a compile would have been paid (20,000 x 100 2.3 s compiled vs 2.9 s eager). TABCTX_COMPILE_AFTER_WARMUP=1 restores compiling them.

Added

  • prod_bench.py --log-calls saves every call with its timings_ms, for timelines of where stalls go.

[0.14.0] - 2026-10-21

Added

  • Where a request's time went (timings_ms on fit and predict responses). A slow call can be waiting (for placement to hear back from a busy worker, for host memory, for the device's lock), moving data (spilling contexts to make room, restoring a spilled one) or computing (CPU preprocessing, the device, compiling a graph), which look the same from outside -- and prod's logs aren't readable from where the stalls are reported. Each request now carries a collector (tabctx/timing.py, a ContextVar) that the engine, the multi-GPU router and the worker processes record phases into: read_table, place, wait_dataset_lock, wait_prepare_gate, prepare, wait_host_memory, wait_device, make_room, fit_device, restore, predict_device, and counts n_graphs_compiled, n_restores, n_predict_chunks. Worker processes send theirs back with each reply. Diagnostic: phases can overlap and keys may change.
  • benchmarks/prod_bench.py: the prod benchmark as a gateway user sees it -- single-user latency (first and repeat fits of new tables per size), mixed load (two users per model looping fit + predicts; tails and calls over 5 s), and throughput (concurrent predicts with per-GPU utilization sampled from the Ray dashboard) -- keeping the slowest calls with their timings_ms. Baseline: baselines/prod-bench-0.13.0.json. benchmarks/gke/serve_bench.sh runs it against Ray Serve in the devbox, configured like prod; devbox.sh also offers 2-GPU machines.
  • Missing values (#61). Inline train_X/test_X cells may be JSON null; CSV cells may be empty, NA, N/A, null, None or NaN. Both become NaN, which both backends handle natively, so callers no longer impute first. On the devbox GPU (serving config), Adult with 30% of cells removed scores the same or slightly better sent as missing than median-filled (TabICL 0.851 vs 0.846 accuracy; TabPFN log-loss 0.320 vs 0.324). Inline tables are converted to float arrays at the HTTP edge (serve/tables.py), which also spares the engine its per-row checks; coalesced predicts concatenate those arrays. Text that isn't a number or a missing value is still a 422, and a CSV row with a blank label is a 422 naming the row. tabicl>=2.2.0 is now required: 2.1.x fails a predict when a training column is entirely missing.

Changed

  • task is optional and inferred from the labels when omitted (#60): non-integer numbers, or whole numbers with more distinct values than both 30 and 5% of the rows (prices, counts: Ames sale prices are integers; 50 state codes in 1,000 rows stay classes), mean regression; anything else means classification, as before. Only actual numbers count: text labels are categories even when they look numeric ("250.01"); a CSV target column, which has no types, counts as numbers when every value parses as one. NaN/inf labels are a 422 whatever the task. FitResponse reports the task used. An explicit task still wins. Engine, multi-GPU router, HTTP API (/v1/tabctx/fit, the legacy predict) and client all take task=None; the client leaves the field out when unset, so it still works against servers that require it.

Fixed

  • TabICL with more than 10 classes failed the fit with a bare 500: its kv cache can't hold the several forward passes tabicl uses past the model's native 10 classes. Such a fit is now a 422 that names the limit, checked before preprocessing once the model is loaded. The legacy /v1/tabicl/predict response now reports the task it fit.
  • A continuous target sent as classification got a 500 from TabICL and a 422 from TabPFN about "573 classes" (#60). Classification labels that are non-integer numbers are now rejected up front with a 422 that says to pass task='regression' (labels.validate_labels); integer-coded classes are unchanged. TabICL's own input errors are now a 422 like TabPFN's, and both backends' rejections of all-numeric classification labels carry the same hint.

[0.13.0] - 2026-10-20

Fixed

  • Predicts against wide or long contexts ran as one-row calls (#47). Predict chunks were sized with the fit memory model, which grows with the training table; past ~3,500 x 500 no chunk "fit", chunking fell to one row, and a 100-row predict became 100 sequential model calls: 14.4 s (TabICLv2) and 6.3 s (TabPFN-3.5) instead of ~0.1 s on an A100, ~24 s and ~11 s on prod's A40s. Chunks are now sized by PredictCostModel (memory/predict.py), built from predict peaks measured under the serving configuration (benchmarks/calibrate_predict.py, A100, compiled TabICL, int8 kv; generated memory/calibration_predict_a100.py): a cost per test row and a fixed cost per call, both by width, plus a cost per training row. It bounds every measurement it was built from and assumes at most linear growth beyond them (affine in width past the widest, where it scales in proportion to width), and it applies only to the calibrated configuration (kv cache, int8, compiled TabICL; TabPFN). When a call's fixed cost alone exceeds the headroom, only the rows above it are sized against the budget instead of splitting to single rows, which only multiplied that fixed cost; with no headroom a predict still runs as the smallest chunks, and a real OOM still halves a chunk. LIGHT_PREDICT_ROWS is now chunking.MAX_CHUNK_ROWS (2,048), so a light predict is one call.
  • A killed worker could look alive to close(), which then kept its host-memory reservations (and failed test_workers_close_together_within_one_grace_period intermittently in CI). The reader thread and close() both reaped a dying worker; when two threads call waitpid at once, the loser finds no child and CPython reports the process still running until the winner records its exit code. Every wait, liveness check and kill of a worker is now serialized under one lock (waits on its process sentinel happen outside it, so health checks never block), so every caller sees the exit, and its code, once one does; a sentinel a forked grandchild still holds falls back to a non-blocking check.
  • TabICL classification labels changed type with return_proba (#45): without probabilities it returned predict()'s labels (numbers for a JSON train_y), with them strings from classes; TabPFN always returned strings. TabICL now derives labels the way TabPFN already did, from one predict_proba pass, and both report them through backends._outcome.classification_outcome: a classification's predictions are always strings drawn from classes. (Integer labels sent as JSON numbers still come back in float form, "1.0"; see #45.) Documented on the predict responses' predictions field.
  • Compiled TabICL stopped compiling in serving (#47). install_compiled_blocks set dynamo's recompile_limit and suppress_errors once, but dynamo's config overrides are per thread (a ContextVar, torch 2.13+), and fits and predicts run on other threads (engine and worker threads, asyncio.to_thread), which saw dynamo's defaults: a limit of 8. Prod's four A40 workers each hit recompile_limit (8) at the first graph past warmup's 54, after which dynamo compiles no new graph of the blocks for the life of the process: every shape warmup missed ran eagerly from then on (warmed shapes kept their graphs), and a compile error would have failed a tenant's fit instead of falling back to eager. The settings are now applied around each compiled call too (tabicl_compile.with_compile_settings, ~4 us), so they hold on any thread, and accumulated_recompile_limit (256, which capped the per-device limit) is raised with them. A100, calls on a separate thread (benchmarks/bench_stalls.py --call-thread; baselines/compile-call-thread-a100-{before,after}.jsonl): shapes outside warmup's families run compiled once their graph exists -- in that sweep, 3,500 x 500 fits 2.31 -> 1.61 s and 3,500 x 1,000 5.49 -> 3.50 s, on graphs first built by its earlier 30k-row fits. Such a graph now compiles once per worker (a first ~30k-row fit takes 5-25 s) instead of never.

Added

  • benchmarks/bench_stalls.py (fixed-cost stalls across table sizes, with compiled-graph counts per call; --tasks, --call-thread to call from another thread as serving does), benchmarks/profile_predict.py (cProfile of one predict), benchmarks/calibrate_predict.py and benchmarks/gen_predict_grid.py (predict calibration), benchmarks/sweep_gateway.py (the table-size sweep through the Vector gateway).

[0.12.0] - 2026-10-18

Added

  • Multi-GPU replicas (#30). MultiDeviceEngine runs one TabctxEngine per device -- backends bound to cuda:i, its own cache, lock and spill subdirectory -- behind the single-engine API, built automatically when a replica sees several GPUs (TABCTX_DEVICES=auto, or an explicit list). Placement is aimed at the expected workload, a few concurrent users with small and large tables: a fit goes where its estimated peak fits without eviction, on the least active device (in-flight calls, then calls in the last 30s), else to the device with the most headroom. Predicts route to the device holding the context (resident first, then spilled), found from lock-free residency/spill snapshots, so routing never waits on another GPU's work and fits and predicts on different devices run concurrently. Per-dataset_id locks serialize fits and evicts of one id (no duplicate copies from retried fits); a re-fit lands on its device before other copies are removed, so a failed re-fit leaves the previous context serving. Unknown models are rejected (422) before anything changes. TABCTX_SPILL_CAPACITY_BYTES is the replica's total, split across devices' spill subdirectories; a spill layout change (one vs several devices) is logged. TABCTX_DEVICES rejects mixed CPU/GPU lists, a bare cuda among indexed GPUs, and indices the process can't see. /readyz gains a per-device breakdown (contexts, bytes, activity, real GPU memory); /v1/tabctx/limits reports the best single device. A single GPU builds exactly the engine it did before.
  • One worker process per GPU (#39, TABCTX_WORKER_PROCESSES, default on for multi-GPU replicas). Fits are host-bound, so threads of one process driving several GPUs contend for the GIL: on 4x A100, four concurrent fits ran at 2.1-3.1x aggregate from threads and 3.96-4.3x from processes (benchmarks/bench_scaling.py). serve/workers.py runs each device's engine in its own process behind a ProcessShard proxy; the router reads versioned residency/headroom snapshots that every reply carries, so its decisions stay lock-free. A dead worker fails its calls, is skipped by placement, and fails /readyz and Ray Serve's check_health so the replica restarts. Nothing depends on Ray. Under Ray Serve on 4x A100 with 6 mixed users: 2.58 -> 5.0 predicts/s, small-fit p50 1.6 -> 0.6 s.
  • Light predicts run beside a fit on the same GPU (#39). A predict against a resident context with at most 2,048 rows no longer waits for a fit on its device (it waited up to 266 s on the L4 rehearsal): ~0.1 s p50 during a 40k x 100 fit on an A100, at ~15% fit slowdown. Restores and large predicts still wait; contexts in use are pinned against evict-ahead (and evicted with predicts held off when nothing else makes room); a fit that runs out of memory beside a predict retries alone; allocator measurements are taken with predicts quiesced, and memory freed during a fit is released only after it has measured itself.
  • TabPFN-3.5 requirement calibration (A100, 20 shapes up to 200k x 10 and 30k x 1,000). TabPFN also sizes its working memory from what is free (20k x 100 peaks at 7.3 GiB unconstrained but runs within 2x its time on 1.1 GiB), so admission bounds resident + working floor for it too, instead of TabICL's peak formula: up to 100k rows at 100 features and 80k at 1,000 on an A40 (was 5k and 500 on an L4). Its per-row resident rate grows with features, which ResidentRateModel(width_dependent=True) respects.
  • Pluggable placement and residency events (#39). The multi-device router asks a PlacementPolicy (placement.py) which device takes a fit, from a lock-free view of each device (headroom, the fit's estimated need there, activity, liveness); the previous rule is the default LeastActiveThatFits, which now also skips dead workers. The router emits ResidencyEvents (events.py) when it places a context and when it removes a copy, to any sink; /v1/tabctx/residency serves the latest. A platform that co-schedules LLMs and tabular models can supply its own policy and follow placements without polling; events are advisory (contexts dropped by capacity eviction without spill aren't reported), so /v1/tabctx/residency also returns each device's current resident and spilled ids to reconcile against, scoped like every /v1/tabctx route (a tenant sees only its own datasets). fit_predict's throwaway contexts aren't reported, and an evict that fails on one device (a dead worker) no longer stops the others.
  • benchmarks/bench_scaling.py (threads vs processes across GPUs), bench_overlap.py (predict latency and correctness during a fit), gke/serve_workload.sh (Ray Serve + workload inside the devbox).
  • benchmarks/bench_workload.py: concurrent users looping fit-then-predict over a small/medium/large table mix (large ones via CSV upload), with per-class latency percentiles, 413s, errors and the per-device spread.
  • benchmarks/sweep_admission.py: the table-size sweep (rows up to the first 413 per feature count, fit and predict timed).
  • ROADMAP: multi-GPU and CUDA-level track (#30-#37), PyTorch-level first.

Changed

  • TabICL's ICL key/value cache is stored in int8 by default (#34, TABCTX_KV_PRECISION=int8|fp16|native; backends/tabicl_kv.py). It dominates a TabICL context, so int8 about halves resident memory per context: twice the contexts per GPU and a longer longest table. A100, 7 datasets (benchmarks/baselines/kv-precision-tabicl-a100.jsonl): identical labels and metrics, max probability drift 0.009, warm predicts up to ~1.4x slower on large contexts. Served probabilities change slightly; set TABCTX_KV_PRECISION=native for 0.11.0's exact outputs. TabPFN already stores its cache in int8.
  • TABCTX_OFFLOAD_DIR (new): where TabICL spills stage outputs too large for GPU and host memory (memory-mapped files) instead of exhausting the pod's RAM; default <TABCTX_SPILL_DIR>/offload, else a temporary directory, one subdirectory per device. Lets one A100 fit 130k x 500 (65M cells).
  • serve.factory.BuiltEngine: backends/backend are replaced by backend_names (with worker processes this process holds no backends), engine may be a MultiDeviceEngine, and close() stops worker processes.
  • TabICL's power transforms are fitted on the GPU (TABCTX_FAST_PREPROCESSING, default on; backends/yeo_johnson.py). tabicl normalizes each ensemble member with scikit-learn's Yeo-Johnson PowerTransformer, which fits one lambda per column with scipy -- 80%+ of a fit's CPU half. A vectorized fit (same objective, bounds and tolerance; grid then golden-section search, all columns at once) runs in 0.08-0.25 s instead of 0.8-4.4 s on an A100 host; lambdas agree with scikit-learn's within one float32 ulp, accuracy/AUC/R^2 are unchanged on the evaluation datasets, and NaN inputs, Box-Cox and a device out-of-memory fall back to scikit-learn. The device section runs under the measurement lock (a concurrent fit's allocator readings never see it) with temporaries bounded to 512 MiB per column chunk, and fitted transformers pickle as plain scikit-learn objects. With compiled blocks, an isolated 40k x 100 fit takes 4.8 s (was 11.2 s).
  • TabICL's attention blocks are compiled (#36, TABCTX_COMPILE, default on; backends/tabicl_compile.py). Inductor fuses the memory-bound layer norms, residual adds and GELU MLP around each attention kernel: on an A100 the fit's GPU half is 1.7-2.2x faster (10k x 50: 0.91 -> 0.52 s; 40k x 100: 7.65 -> 4.30 s; 20k x 200: 6.51 -> 2.92 s) and its peak ~20% lower (40k x 100: 33.1 -> 27.0 GiB); 100-row predicts 20-35% faster. Accuracy, AUC and R^2 are unchanged on the evaluation datasets, and predicts beside a fit stay correct. Ray Serve, 6 mixed users on 4x A100: 5.78 -> 6.99 predicts/s, small-fit p95 1.48 -> 0.95 s. tabicl's ssmax log(n) is now computed symbolically inside compiled code, so a new table size doesn't recompile. Each worker's compile pool is sized to its share of the container's CPUs (TORCHINDUCTOR_COMPILE_THREADS, when unset). Only tabicl's fp16 path (1,024+ rows) is compiled; its common graphs (~50) compile in warmup() at replica start (~130 s per worker, workers in parallel); shapes it doesn't cover (very large tables) compile on first use, ~10 s once per worker. Compile errors fall back to eager, with a warning after warmup if nothing compiled. The lower peak doesn't lower what admission must reserve: the least free memory a compiled fit succeeds with is the same as eager's or up to ~2 GiB more (80k x 10: 22.9 vs 20.7 GiB), so compiled TabICL is admitted against its own requirement grid (per shape, the larger of the two measurements; minfree-tabicl-kv-compiled-a100.jsonl).
  • Host-memory admission (new; memory/host.py). Large TabICL fits offload stage outputs to host RAM (~46 GiB at 10M cells); two at once could get the pod OOM-killed with every tenant's context. Each backend estimates a call's host use (TabICL ~5 KB per cell when it offloads, else nothing; TabPFN ~0.6 KB per cell), and a guard admits calls against one budget shared by every worker process on the node: first come first served (a large fit can't be starved by modest ones), at most slots offloading calls at once (each budgets 1/slots of free host memory for its staging), calls under 1 GiB never wait. The budget is the container's memory limit x 0.85 minus its working set (every process in the pod) and an 8 GiB reserve, re-measured once the per-GPU workers have warmed up. A worker that dies holding host memory has it returned by its parent; a stuck accounting lock fails calls with HostMemoryUnavailableError (new, exported; HTTP 503, retryable) instead of hanging. /readyz gains host_memory (budget_bytes, reserved_bytes, calls_active, calls_offloading, calls_waiting, large_fit_slots).
  • TabICL fits prepare their CPU half before taking the device: preprocessing (~30-37% of a fit) overlaps another fit's GPU half -- back-to-back fits 17-26% faster on an A100. At most one fit per device prepares ahead, so queued fits don't each hold preprocessed copies in host memory.
  • Worker processes shut down together under one 10 s deadline (was up to 10 s each, past a pod's grace period).
  • tabicl's per-call log(n) host-to-device syncs are cached (bit-identical): 10k x 50 fits 1.98 -> 1.66 s on an A100.
  • The cache's spill reads and writes run outside its lock, so bookkeeping and light predicts never wait on multi-GB disk I/O.
  • TabPFN contexts share the model: ~42x less GPU memory per context. TabPFN builds a fresh model for every estimator unless its built-model cache is on, so every cached context held its own ~850 MB copy of the weights on the GPU (measured on an A100: 4 contexts of a 300x6 table took 3.4 GB). TabPFNBackend now enables that cache (TABPFN_MODEL_CACHE_SIZE, default 16 entries, keyed by device and task): one shared model per (device, task), about 1.7 GB fixed per GPU, and a context costs only its own state (~20 MB for that table). Interleaved contexts predict bit-identically to each context alone.
  • TabPFN spill blobs no longer contain the model. A spilled TabPFN payload was ~900 MB, mostly weights, and a restore built another private copy on the GPU. References to the shared model are now pickled as tokens and re-attached on load: a 2,000x20 context spills as 99 MB and restores with identical predictions and no extra GPU memory. Older blobs still load.
  • Weights load at replica start (warmup() on both backends, called by the factory) instead of inside the first tenant's fit, which reported the checkpoint as part of its context (a 200x6 TabPFN context was booked at 855 MB, TabICL's at 226 MB) and paid the load latency.

Fixed

  • Fits OOMed on GPUs the books said had room (4x L4 rehearsal: a 250 x 1000 TabPFN fit beside 18 GB of contexts on a 22 GB card). Memory outside the cache's accounting grows after warmup (allocator fragmentation, compiled kernels' workspaces), so making room now also checks the device's real free memory (free plus torch's reusable cache) and evicts until both agree. A fit retrying after an OOM first frees contexts that were parked during its failed attempt.
  • Router 404 for a context being spilled (4xL4 rehearsal: a fit returned 200 and the next predict 404'd 27 ms later): a context moving between the GPU cache and the spill tier was listed by neither; it is now tracked in transit, and predict re-locates once after a miss.
  • TabICL shared-backbone cache was per-module, not per-call: interleaved calls could read another context's cache. It is now thread-local.
  • A device OOM while restoring a spilled context no longer deletes the spilled copy.
  • First fit on a GPU failed with "Invalid device argument": device-explicit memory-stat calls need CUDA initialized, which the no-argument forms used to do implicitly. Backends now initialize CUDA at construction.
  • Spill tier warm restart: the index rebuilt at startup was keyed by the hex-encoded file name instead of the dataset_id, so lookups, explicit deletes and disk-pressure cleanup all missed contexts a previous process had spilled (restores still worked because load() reads by path).
  • TabICL spill restore reloaded the checkpoint: tabicl's __setstate__ calls _load_model(), which loaded the full checkpoint (and a second backbone onto the GPU, unaccounted) on every restore before tabctx swapped in the shared one. Spilled TabICL payloads now carry a task tag and restore straight onto the shared backbone; untagged blobs from older versions still load.
  • /readyz and the legacy endpoint read GPU memory from torch's current device; they now read the replica's own device(s).
  • The TabICL and TabPFN backends measured memory on, and let library internals allocate on, torch's current CUDA device rather than their own. Harmless on one GPU; on cuda:i it would have charged every context to cuda:0. They now measure on their device and run inside its scope.

Validated

  • Unit (multi-device placement, routing, spill restore, failures) and integration tests (the serve app on two fake devices through Ray). Concurrency tests force interleavings with gates and barriers rather than timing: a fit on an idle device completes while another device is mid-call, lookups and stats never wait on a busy device, concurrent same-id fits leave one copy, a predict during a re-fit is answered. Seven of them fail against the first version of this engine (found by review).
  • On a local fake 3-device replica with 5 concurrent users: with spill on, 0 failed predicts at ~100 predicts/s. Without spill, capacity eviction cost 12 predicts on three devices versus 115 on one.

[0.11.0] - 2026-10-17

Changed

  • The TabPFN model id follows the checkpoint actually loaded. It was hardcoded to tabpfn-3, but tabpfn 9.x loads TabPFN-3.5 by default (a different model under its own license, tabpfn-3-5-license-v1.0), so a default install served TabPFN-3.5 named tabpfn-3. The id is now derived from tabpfn's resolved model_version (TABPFN_MODEL_VERSION): v3.5 -> tabpfn-3.5 (the 9.x default), v3 -> tabpfn-3, v3.5-fast -> tabpfn-3.5-fast. Breaking for callers that send "model": "tabpfn-3" to a deployment on the default version: pin TABPFN_MODEL_VERSION=v3 to keep serving TabPFN-3 under its old id, or switch callers to tabpfn-3.5.

Fixed

  • The TabPFN pretraining-limit integration test used 2,100 features, which TabPFN-3.5 accepts (its feature limit is 20,000, versus 2,000 on v3). It now exceeds the 160-class limit shared by every v3-series checkpoint.

Validated

  • tests/integration/test_tabpfn_backend.py on CPU with tabpfn 9.1.0 against both v3 and v3.5, offline (HF_HUB_OFFLINE=1, no TABPFN_TOKEN) from a read-only TABPFN_MODEL_CACHE_DIR holding only the checkpoint: 4/4 each. TabPFN-3.5 needs one file, tabpfn-v3.5-20260909.safetensors (835 MiB), for both classification and regression.

[0.10.0] - 2026-09-01

Fixed

Found by a full-codebase review (2026-09-01), each with a regression test: - Explicit eviction could resurrect a stale context. ContextCacheManager.evict() (fit_predict's cleanup, re-fit overwrites) dropped the resident entry but left any spilled copy on disk, so the next get() transparently reloaded the OLD context. Explicit evict and re-fit put() now remove the spilled copy too. - Spill restore under a smaller budget was a 500. A context spilled by a replica with a larger TABCTX_GPU_MEMORY_FRACTION failed re-admission with CacheCapacityError from inside get(); it is now a clean miss (re-fit), and predict maps admission errors to 413 as fit already did. - predict() raced evict-ahead-of-fit. The context lookup and chunk-size derivation ran outside the cache lock, so a concurrent fit could spill the context and shrink the headroom between lookup and GPU call. The whole sequence now runs under the lock (RLock, so nothing else changes). - The serving budget was always an A100's. serve/factory.py hardcoded 40GB device capacity; on an L4 (24GB) admission headroom exceeded the card. Capacity is now read from the visible CUDA device (detect_gpu_capacity_bytes), the cache ceiling keeps its calibrated 24/40 ratio to it, and /readyz + /v1/tabctx/limits report gpu_capacity_bytes. - Client: GET calls (ready, models, limits) leaked raw urllib errors instead of tabctx exceptions; upload_csv_file read the whole file into memory despite existing for tables too big for inline JSON (it now streams the open file with an explicit Content-Length, verified against a real HTTP server in the unit suite); the affinity header name is configurable (session_header=) to follow a deployment's RAY_SERVE_SESSION_ID_HEADER_KEY, without which affinity silently degraded. - Factory default model was chosen by comparing a backend kind (tabicl) against model ids (tabicl-v2) -- correct only because dict insertion order happened to agree. Now explicit.

Changed

  • CachedContext.feature_names: a CSV-fit context's column names live on the context (and in its spill metadata) instead of a per-replica dict in serve/app.py that grew without bound and lost the schema check whenever a context was restored from spill or a replica restarted. TabctxEngine.fit(feature_names=...) / engine.feature_names(dataset_id).
  • Cache usage is tracked incrementally (make_room polled an O(n) sum per eviction); ContextCacheManager.dataset_ids() added. Estimator/factory defaults reference the module constants they duplicated; adaptive observations use a bounded deque.
  • Housekeeping: .env ignored; ruff no longer formats Markdown code blocks; README architecture figure (docs/figures/architecture.svg), status section, and a "pre-computed contexts for many applications" section stating what works and what's missing (ROADMAP item 1).

Added

  • benchmarks/probe_deployment.py: end-to-end acceptance probe for a live deployment on any GPU via TabctxClient (tenant + custom session header aware): cache reuse, single-pass proba, upload/fit/predict by reference + schema check, chunked predicts, coalescing, and fill-until-eviction with spill restore.

Validated

  • All of the above on a live GKE deployment (tabctx-gateway, one NVIDIA L4 24GB, TabICLv2 + TabPFN-3, spill on) via probe_deployment.py: 24/24 checks. The device-capacity fix is visible there: cache ceiling 14.3GB / admission headroom 20.1GB (was 24GB / 36GB, i.e. more than the card). Deployment gotcha recorded for next time: Ray caches pip runtime envs by package spec on every node, and the Serve controller ships the ingress-wrapped class from the HEAD's cached env by value -- a same-named wheel only goes live after the head pod (not just the worker) restarts.

[0.9.1] - 2026-08-29

Fixed

Two bugs found by the dedicated GKE stress run (a fresh single-replica A100 deployment of v0.9.0 with both models and spillover): - One oversized request could flush every tenant's context. Evict-ahead-of-fit drained the ENTIRE cache (spilling 30 contexts) making room for a 200k x 60 fit that could never be admitted, then 413'd anyway -- observed live via /readyz (cache 0 contexts, spill tier at capacity). Feasibility is now checked against the empty-cache headroom BEFORE any eviction: infeasible fits reject immediately and warm contexts stay put. - Predicts against calibration-admitted shapes degraded to 1-row chunks. Chunking still estimated with the stale conservative formula against the cache ceiling, so a 2,000-row predict on a 50k x 100 context became 2,000 sequential GPU calls and timed out (the shape 413'd instantly before v0.9.0, so this path had never run). The calibration sweep's measured predict-time peaks now ship as a second preloaded grid (*_PREDICT_PEAK_GRID, scaled linearly in n_test from the 1,000-row measurement basis), and chunking budgets against the same usage-aware headroom admission uses.

[0.9.0] - 2026-08-29

Changed

  • Standard multi-model endpoint (chat-completions style). One deployment now serves several models over ONE shared context cache and GPU budget: TABCTX_BACKEND=tabicl,tabpfn (first listed = default), requests select a model via "model": "<exact id>" (tabicl-v2, tabpfn-3 -- named for the checkpoints actually loaded), GET /v1/models lists what's served. Contexts remember which model fit them, so predict dispatches automatically; a single shared budget is the point -- two separate deployments on one GPU would each be blind to the other's memory use. Single-model deployments are unchanged. Requires Python >= 3.12.
  • Admission control now bounds the right quantity, measured on real hardware (the former Priority-1 rewrite):
  • Backends report TWO measurements per fit: the transient PEAK allocation (what actually OOMs -- can exceed the resident context by an order of magnitude) and the post-fit RESIDENT size. Admission learns from peaks; cache accounting charges residents. (v0.4.0 conflated these -- admission was learning from residents.)
  • Admission is usage-aware: a fit's estimated peak must fit within 85% of device capacity MINUS what the cache already holds resident, so the same shape honestly stops being admissible as the replica fills and recovers after eviction. /v1/tabctx/limits reports the live boundary.
  • AdaptiveMemoryEstimator ships with a factory-preloaded calibration grid (memory/calibration_tabicl_a100.py, generated by benchmarks/calibrate_memory.py + gen_calibration_data.py from an A100-40GB sweep of TabICL with the kv cache on): measured peaks for shapes up to 100,000 rows x 200 features in "kv" mode and 500,000 rows in "repr" mode (whose contexts are ~10x smaller resident: 1.6GB vs 14.8GB at 100k x 100 -- the high-tenant-density option), with the real boundaries recorded: ~200k rows GPU-OOMs in kv mode, and ~10M-cell tables (200k x 100, 100k x 200) exhaust ~85GB of HOST RAM in the model's CPU-side pipeline -- pod memory limits are the backstop for that. Calibration points carry a 1.1x margin (direct measurements) vs 1.5x for runtime observations. Net effect, validated end to end on an A100: a 100,000 x 10 fit -- rejected by every prior version -- admits and completes in 17s, and fits beyond the measured boundary still 413 cleanly.
  • Disk spillover tier (TABCTX_SPILL_DIR, off by default): capacity-evicted contexts serialize to local disk and reload transparently on the next request -- no training data needed from the caller; bounded by its own LRU (TABCTX_SPILL_CAPACITY_BYTES), best-effort by contract, warm-restart capable. TabICL spills exclude the shared pretrained backbone. Validated under real pressure on an A100 (3 contexts / 39GB spilled, evicted context reloaded with correct predictions). Measured economics are in the module docstring: on slow disks a reload can lose to a re-fit -- enable it for fast NVMe, expensive fits, or callers who can't re-send data.
  • Evict-ahead-of-fit (found by the OOM it prevents, on a real A100): when a fit's estimated transient peak exceeds current headroom, cold contexts are evicted/spilled until it fits, instead of rejecting a fit that is safe once they step aside. Admission now runs under the cache lock (no two fits can pass on one snapshot), and the transient budget is capped at 85% of device capacity -- a fit admitted with ~400MB nominal slack at 90% still OOMed to allocator fragmentation.
  • Fixed a multi-GB eviction leak: tabicl's forward_with_cache leaves a pointer to the active context's kv cache on the (shared) backbone module, so the most recently used context stayed pinned on the GPU after tabctx evicted it -- ~15GB the accounting said was freed but never left the device, eventually OOMing a fit the books called safe. The backend now clears the module pointer after every fit/predict.
  • Cross-context batching: investigated, mapped, and deliberately deferred with the full feasibility map preserved in batching.py's docstring (same-shape wrapper-level batching is exact but narrow; cross-shape needs upstream model changes and has a silent-accuracy trap in SSMax). Same-context coalescing remains the batching layer; real-user traffic will decide whether the narrow case is worth building.
  • Tooling: Python >= 3.12; ruff + pre-commit + uv lockfile; CI split into code checks / unit tests (with coverage) / integration tests; publish.yml (PyPI Trusted Publishing, pypi environment). README's "Why" section rewritten for first-time readers.

[0.8.0] - 2026-08-29

Added

  • Large-table ingestion: upload, then fit/predict by reference (ROADMAP Priority 3, inspired by comparing API shapes with PriorLabs' hosted service). New POST /v1/tabctx/upload streams a raw CSV to replica-local disk (never buffered in memory, size cap enforced mid-stream via TABCTX_MAX_UPLOAD_BYTES, TTL expiry via TABCTX_UPLOAD_TTL_S); fit accepts train_upload_id/target_column and predict accepts test_upload_id as alternatives to inline JSON. Uploads are single-use, tenant-scoped like dataset ids, and replica-local -- correct at num_replicas >= 2 because the same x-session-id affinity contract routes upload, fit, and predicts to one replica. Test-CSV headers are checked against the training schema (reordered columns 422 rather than silently mispredicting). CSV parsing (serve/csv_io.py, numpy C parser, float32) and the store (serve/uploads.py) are transport-agnostic modules with their own unit tests; the engine is unchanged apart from an O(1) rectangularity fast path for 2D arrays (replacing a per-row Python loop that would cost real time at 10^6 rows). TabctxClient gains upload_csv/upload_csv_file/fit_uploaded and predict(test_upload_id=...). New errors: UploadNotFoundError (404), UploadTooLargeError (413). Transport and admission remain separate layers: a huge CSV uploads fine and the memory-admission gate then rules on the table itself.
  • TabPFN backend (backends/tabpfn.py, TABCTX_BACKEND=tabpfn, pip install tabctx[tabpfn]): the second real backend behind the TabularICLBackend protocol -- engine, cache, estimator, and serving stack all unchanged, which was the design claim. The TabICL kv-cache lesson applies verbatim: TabPFN's default fit_mode="fit_preprocessors" still re-runs the training-context forward pass per predict, so tabctx maps TABCTX_KV_CACHE onto it (kv->fit_with_cache, repr->fit_preprocessors, off->low_memory). TabPFN's pretraining-limit rejections surface as 422s, CUDA OOM as BackendComputeError. Validated on CPU with the real model (license portal: ux.priorlabs.ai + TABPFN_TOKEN): engine-stack roundtrip, cache-mode prediction equivalence (labels identical; probabilities drift up to ~1e-2 between fit_with_cache and fit_preprocessors due to TabPFN's reduced kv_cache_precision -- documented in the test), regression, and clean 422s past the ~2,000-feature checkpoint limit. GPU validation and TabPFN-specific memory calibration still pending (estimator calibration is TabICL data).
  • /v1/tabctx/limits + TabctxClient.limits(): capability discovery (inspired by PriorLabs' get_model_limits) reporting the live, adaptive admission boundary -- max admissible train rows at representative feature counts -- so clients validate shapes before uploading tables.
  • benchmarks/bench_vs_hosted_api.py + baselines/v0.7.0-vs-hosted-api-cpu.json: serving-architecture comparison against PriorLabs' hosted TabPFN API on the fit-once-predict-many workload. tabctx warm predicts are 6-14x faster at p50 even with the local side on a laptop CPU (each hosted predict pays upload + network round trips); the hosted side wins cold-fit on larger tables absent a local GPU. Explicitly NOT a model-speed comparison -- the baseline file and script both say so.

[0.7.0] - 2026-08-29

Added

  • tabctx.client.TabctxClient: pure-stdlib HTTP client that makes the serving contracts impossible to get wrong -- it sends x-session-id (== dataset_id, required for correct multi-replica routing) and x-tabctx-tenant-id automatically, maps HTTP errors back to the same tabctx exception types the in-process engine raises, and retries 503 backpressure with exponential backoff (safe by construction under strict affinity). End-to-end tested against the local 2-replica deployment.

Performance

  • TabICL's kv-cache is now enabled by default (TabICLBackend(kv_cache="kv"), env-tunable via TABCTX_KV_CACHE=kv|repr|off). tabicl defaults it OFF, and without it every predict() silently re-encodes the ENTIRE training set through all three transformer stages -- i.e. the exact repeat cost tabctx exists to eliminate. Found by tracing tabicl's predict_proba internals (transform(mode="both") re-concatenates train+test per call; with the cache, test rows only attend to fit-time-cached K/V). Verified on CPU that kv-cached predictions are identical to the uncached path (max probability diff ~1e-6) and warm predict is 2.9x faster even for a tiny 40-row context; the win grows with training-set size since the avoided re-encode is O(n_train^2). The cache's GPU memory cost lands inside fit()'s measured allocation delta, so cache accounting and the adaptive admission gate price it automatically.
  • Pretrained backbone weights are loaded once per process and shared across fits instead of torch.load + H2D-transferring hundreds of MB per fitted context (5.8x faster re-fit on CPU). Uses tabicl's own sanctioned sharing pattern (_unsupervised/_finetune do exactly this). Safe because the engine serializes every backend call behind its cache lock, so the shared module's per-call mutable state is never touched concurrently.
  • Same-context predict coalescing (batching.py, CoalescingPredictor): concurrent requests against one cached context are packed into a single backend call within a small window (TABCTX_BATCH_WINDOW_MS, default 5ms; 0 disables) and results are split back per caller. One malformed member is retried individually so it fails alone (422) without poisoning batchmates. Cannot exceed the memory budget: the engine still runs exactly one GPU call at a time and chunks oversized merged batches. /readyz reports coalescing counters.
  • max_ongoing_requests raised 2 -> 8 (and max_queued_requests 8 -> 16): safe since GPU work is serialized by the engine lock regardless; more in-flight requests just means more coalescing opportunity.

[0.6.0] - 2026-08-28

Fixed

  • Multi-replica deployments are now correct (previously the architecture was only safe at exactly one replica -- ROADMAP.md's former Priority 1). The Ray Serve deployment now ships with Ray's consistent-hash request router (ray.serve.experimental.consistent_hash_router.ConsistentHashRouter, requires ray >= 2.58) configured for strict session affinity (num_fallback_replicas=0): requests carrying the x-session-id header always land on the same replica. The contract, enforced in the new serve/affinity.py, is that the session id IS the dataset_id -- clients send x-session-id: <dataset_id> on every /v1/tabctx/fit and /v1/tabctx/predict call (fit adopts the header value as the dataset_id when the body omits one; a mismatched header/body pair is rejected 422 loudly rather than silently mis-routing). Requests without the header behave as before, which only routes correctly on single-replica deployments.

Security

  • Tenant isolation (former ROADMAP.md Priority 2): the serving layer now scopes every dataset_id by the caller's x-tabctx-tenant-id header before it touches the cache (serve/tenancy.py), so a guessed/shared dataset_id no longer reaches another tenant's cached model. TABCTX_REQUIRE_TENANT=true makes the header mandatory (401 without it), eliminating the unscoped namespace entirely; the default keeps the header optional for dev/backward compatibility. Tenant identity is caller-supplied and unverified by design (pair with an authenticating proxy for real security -- trust model documented in the module). Malformed tenant ids are rejected 422 rather than silently unscoped; responses never leak the internal scoping prefix.

Added

  • serve/factory.py: environment-driven engine construction, shared by any host. TABCTX_BACKEND=fake runs the full serve app against the deterministic fake backend (no GPU/torch), which is what makes multi-replica routing testable on a laptop and in CI. TABCTX_GPU_MEMORY_FRACTION scales the estimator's admission ceiling and the cache's capacity budget for replicas sharing one physical GPU (e.g. two replicas at num_gpus: 0.5 on one A100).
  • served_by (replica tag) in fit/predict responses and replica in /readyz, so clients and probes can verify affinity actually pinned a tenant's traffic to one replica.
  • tests/integration/test_multi_replica_affinity.py: boots a real local 2-replica Ray Serve cluster (fake backend) and asserts 16 datasets x 6 predicts see zero spurious 404s, every predict lands on its fit replica, and fits spread across both replicas. This is the regression test that would have caught the routing gap immediately; also run in CI (new integration job).
  • benchmarks/bench_concurrency.py now sends the session header on all tabctx-native calls, so its numbers stay valid at num_replicas >= 2.

[0.5.0] - 2026-08-28

Fixed

  • Malformed input (mismatched train_X/train_y lengths, empty tables, ragged rows, zero-feature rows, a predict() feature count mismatched against the cached context) now raises InvalidInputError -> HTTP 422, instead of reaching the backend as raw numpy/sklearn arrays and surfacing as an unhandled, untranslated exception (a bare 500 with no useful detail). Found via probe_scale.py's malformed-input test against a real deployment.

Context

Found during a dedicated race-condition/throughput/input-validation test pass (tests/gke-tabicl-test/probe_scale.py in inference-platform), run to validate the service is robust to one careless client's bad request -- a real multi-tenant requirement, not just a nice-to-have. That same pass confirmed no data corruption under concurrent fit()/predict() races on the same dataset_id, and measured real sustained throughput (~1.2 fit+predict cycles/sec across 8 concurrent users, p95 latency ~7.7s) -- concrete evidence of the coarse per-replica lock's real-world cost under load, which is already a documented gap.

[0.4.0] - 2026-08-28

Added

  • AdaptiveMemoryEstimator (memory/adaptive.py): wraps the static PowerLawMemoryEstimator as a fallback, but the pre-fit admission gate now uses real per-fit measurements (fed back via engine.fit() -> record_observation()) whenever a past fit at least as large in both rows and features has been observed -- safe by construction (memory cost is assumed monotonic in table size, so a real measurement on a larger-or-equal shape is a valid upper bound for a smaller query), with a configurable safety margin on top. Falls back to the static formula for genuinely novel or larger-than-anything-seen shapes, and always for predict()-time chunking queries (a different, unmeasured quantity). Deployed as the default estimator in serve/app.py.
  • MemoryEstimator protocol gained record_observation(); PowerLawMemoryEstimator implements it as a no-op.

Context

Direct extension of the v0.3.0 cache-accounting fix: now that fit() calls report real measured GPU cost, this closes the loop by feeding that data back into the PRE-FIT admission gate too, so admission decisions get progressively less conservative for shapes the service has actually served -- compounding the multi-tenant capacity improvement from v0.3.0 over the life of a running replica.

[0.3.0] - 2026-08-28

Fixed

  • Cache-accounting size for a fitted context now comes from the backend after fit() runs, not before. TabICLBackend now reports the real torch.cuda.memory_allocated() delta measured across fit() instead of the pre-fit formula-based estimate. Found via extensive multi-tenant load testing on a real A100-40GB: the formula-based estimate was ~14x higher than real GPU memory for a realistic shape (21.85GB accounted vs 1.56GB actually resident), needlessly throttling effective multi-tenant cache capacity to a fraction of what the hardware supports.
  • The pre-fit admission-control gate (which must stay conservative, since nothing has run yet to measure) is unchanged -- this fix only affects how much of the cache's capacity budget a successfully fitted context is charged.

Context

Found and fixed during an extensive multi-tenant test suite (concurrent users, cache eviction under pressure, leak detection) run against a real GKE A100-40GB deployment, in direct response to a requirement that this service support several concurrent users at scale.

[0.2.0] - 2026-08-28

Added

  • /readyz now reports real GPU memory (torch.cuda.memory_allocated/memory_reserved) alongside the cache's own byte-accounting, so the two can be compared to catch the estimator drifting from reality or evicted contexts not actually releasing GPU memory.

Context

Added while running an extensive multi-tenant test suite (concurrent users, cache eviction under pressure, regression task, dataset_id reuse, repeated fit/evict cycles) against a real GKE A100-40GB deployment, per the requirement that this service support several users at scale and be tested accordingly.

[0.1.0] - 2026-08-28

Initial release. Multi-tenant context caching engine for tabular ICL models (TabICL backend), calibrated memory estimator, estimator-driven test-row chunking, and a Ray Serve deployment. See README.md for full v1 scope and known gaps.