Nums AI Releases Causilo, a Tabular Foundation Model Topping TabArena Among Single Models

Causilo is deployable for research and evaluation today on CUDA or CPU, but commercial, production and hosted API use require a separate license from Nums AI.

Causilo is an in-context learning model. Calling fit does not update the pretrained weights. It stores the training rows as context and predicts query rows in one forward pass. According to its TabArena submission, Nums AI pretrained Causilo only on synthetic data, with no TabArena datasets. Inputs can be NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification supports up to 10 classes. Regression returns mean predictions by default. Version 1.0.1 adds median and quantile outputs, based on 999 native quantiles.

Nums AI splits the network into three phases: refinement, compression and in-context learning. Features are grouped in sets of three. Each value is embedded with 16 learned sine and cosine frequencies. Missing values get their own learned vector. Two column stages summarize each feature group. In each, 128 latent slots read only the training rows and pass that summary to every row. Between the two column stages, a row stage lets feature groups interact through four latent tokens. It uses cross-attention instead of full self-attention, which Nums AI says keeps cost linear in feature count. A pooling block then compresses each row into a fixed 512-dimensional vector. Labels are added to the training rows. A 12-layer prediction block lets query rows attend to those labeled rows. Query rows cannot change the training context or each other. By default, eight ensemble members share the same weights. Each one cycles through none, rank2gaussian, robust or power normalization, with seeded feature and class permutations.

For TabArena, Nums AI used the official pipeline: 51 datasets and 816 Full splits, with eight estimators and seed 42. A TabArena maintainer re-ran the full evaluation and got the same overall Elo of 1794, while the table lists Causilo's overall Elo at 1792.9. In classification, Causilo's Elo is 1771.8, compared with 1758.8 for EXAONE Tabular, with an improvability of 0.0875. In regression, Causilo's Elo is 2032.6, compared with 1992.8 for TabFM, with an improvability of 0.0125. The overall next best single model is TabFM at 1764.4, with an improvability of 0.0684. The field includes Google Research's TabFM, LG AI Research's EXAONE Tabular and Prior Labs' TabPFN-3 at 1636.2 overall.

Several qualifications accompany these numbers. The number one positions exclude system entries. With systems included, the maintainer re-run placed Causilo third of 88 overall. On improvability, TabFM still leads overall and on classification, while Causilo leads on regression. The Elo confidence intervals at the top overlap, so the lead over TabFM and EXAONE Tabular is narrow. Nums AI also lists Xiaomi-TabLDM and Amazon's Mitra-v2 behind Causilo. Neither model appears in the benchmark files in Causilo's repo.

On ScoringBench, which scores regression models with proper scoring rules such as CRPS alongside RMSE and R-squared, Nums AI submitted Causilo 1.0.1 on 101 datasets, five folds each, capped at 3,000 samples. Nums AI reports that Causilo ranks first by CRPS, R-squared and RMSE. The ScoringBench maintainer independently checked the results before committing them.

Nums AI also reran three models on one H100 80 GB GPU, with eight CPU cores per job. Causilo recorded 2.504 seconds fit time per 1,000 rows, 0.251 seconds predict time per 1,000 rows and 8.15 GiB GPU memory. TabICLv2 recorded 3.449 seconds, 0.303 seconds and 8.37 GiB; TabPFN-3 recorded 4.18 seconds, 0.686 seconds and 0.88 GiB. In this test, Causilo was fastest on both fit and predict, while TabPFN-3 used far less GPU memory. Setting use_kv_cache=True moves context work into fit, using more memory to speed up repeated predictions.

Causilo needs Python 3.10 to 3.12 and PyTorch 2.13 or newer. The first fit downloads the checkpoint automatically. Basic usage imports CausiloClassifier and CausiloRegressor from the causilo package. The classifier can be configured with n_estimators and random_state; predict_proba returns class probabilities. The regressor can return quantiles with output_type set to quantiles and a list such as 0.05, 0.5 and 0.95. A Hugging Face demo Space is also available.