AI in Scientific Research: How to Tell Where It Is Actually Working

“AI discovered a new material.” “AI found a drug candidate.” “AI solved protein folding.” Each of those sentences can be true, badly misleading, or flatly wrong depending on one thing the sentence does not tell you: how far the result got from the model before somebody wrote it down. The sentence that hides four different claims Take a single headline: a model proposed a molecule that binds a protein implicated in a disease. That sentence is compatible with at least four very different states of the world. The molecule might exist only as a string in a file. It might have been synthesised. It might have bound the protein in a test tube. Or it might have improved an outcome in a person. Those four are separated by years, by orders of magnitude in cost, and by a probability of success that drops at every step — and press coverage routinely reports the first as though it were the fourth. This is not a complaint about journalism. It is the single most useful thing to internalise about the whole field, because once you have the ladder in your head you can grade a claim in about ten seconds, and you can do it for a subject you know nothing about. The ladder The rungs are the same in every discipline. Only the names of the instruments change. Rung Description 1 · Output The model emitted something: a structure, a score, a candidate, a forecast. Nothing has been checked. Everything downstream is conditional on this being worth checking. 2 · Retrospective The output was compared against data that already existed — held-out structures, historical weather, known compounds. This is where nearly all published numbers live, and it is entirely dependent on the held-out set resembling the future. 3 · Prospective The prediction was made first and the answer arrived afterwards. A forecast verified against what the weather then did. A candidate synthesised after being proposed. This rung is qualitatively stronger than rung 2 and much rarer. 4 · Confirmed An independent method establis