Why LLMs Still Struggle With Tabular Prediction

Most business prediction problems do not arrive as prose. They arrive as rows: account attributes, transactions, sensor readings, test results, and a target column. For this kind of data, gradient-boosted trees and other conventional methods remain hard to displace. A new paper, Why Large Language Models Fail at Tabular Prediction , asks a much more useful question than “can an LLM classify a table?”: what, specifically, breaks as the task becomes more like ordinary tabular machine learning? The answer from the authors’ controlled experiments is input dimensionality. Their result matters because it separates a real limitation from several explanations that sound plausible but did not hold up in their tests. The experiment was about prediction, not table chat The paper evaluates frontier LLMs in a pure inference setup: a model receives labeled examples and must predict labels for new rows in a single generation pass. There is no fine-tuning, retrieval pipeline, tool calling, or agent loop to compensate for the base model. This is deliberately narrow. It asks whether a general-purpose language model can act as a direct tabular learner. Across 31 benchmark datasets, the authors compare nine methods and 252 configured classical models. That scope is important: a weak result on one CSV is easy to explain away as prompt design or a quirky dataset. A consistent trend across many tasks is harder to dismiss. The headline is not simply that LLMs lose to established tabular baselines. It is that their accuracy declines as the number of input dimensions grows, while the classical baselines in the study stay stable or improve. The paper therefore treats dimensionality as the central failure mode rather than an incidental property of difficult datasets. Four popular explanations did not survive testing There are several standard reasons developers give for poor LLM performance on tables. The researchers turn these into falsifiable hypotheses. “The classes overlap too much.” If th