Data Analysis With LLMs: Where It Breaks

Ask a model to analyse a dataset and it writes code, the code runs, real numbers come out, and a paragraph explains what they mean. Three independent things had to be right. Only one of them tells you when it was not. Three places to be wrong The code can be wrong. If it crashes you find out immediately, which is the benign case. The dangerous case is code that runs cleanly and computes something other than what you asked. The statistics can be wrong. The code faithfully executes a procedure whose assumptions the data violates, or which answers a different question from the one you have. Nothing errors; the number is simply not evidence for what you think. The interpretation can be wrong. This is where the model is on its home turf and at its most dangerous, because generating a fluent explanation of a result is exactly what it is good at, and it will do so with equal confidence whether the result supports the explanation or not. Code that runs and is wrong A short list of things that produce no error and change the answer. Every one of them is ordinary and none is specific to models — but a human writing the code usually knows the dataset, and the model does not. Silent row loss. Missing values dropped by default somewhere in the chain, so the analysis runs on a subset that is not random with respect to the outcome. Joins that change cardinality. A merge intended as one-to-one that is actually many-to-many, silently duplicating rows and inflating every count and every significance test downstream. Type coercion. A column read as text because of one stray value, then coerced to numbers with the failures becoming missing values that get dropped by the previous bullet. Grouping that discards keys. Missing group labels dropped by default, so an entire category disappears from a breakdown without appearing anywhere in the output. Units and encodings. A column the model assumed was a percentage and is a proportion; a sentinel value like -999 treated as a measurement; a d