Beyond LLMs: How World Models Are Changing Generative Media

If I give my daughter milk or water, I can almost guarantee she will spit it out. She’s 21 months old, and while she actually enjoys both drinks, halfway through she inevitably decides it’s time for a daily physics experiment. She wants to know: What will happen if I open my mouth and just let the water fall out? How Humans Learn the World While it annoys me, I remind myself that this is how humans learn about the world: by building a mental map through repetition and cause-and-effect. Over time, these experiences allow our brains to predict outcomes without needing to test them every single time. For example, I don’t need to jump off a multi-story building to know I’ll get hurt. I can infer the outcome from smaller falls or by observing what happens to other people and objects. AI researcher Yann LeCun describes this kind of understanding as an "internal world model": a mental representation of how the world works that helps us predict what might happen before we take an action. LLMs vs. World Models When most of us think of generative AI, we immediately associate it with large language models (LLMs) because that is what we have been using heavily for the past few years. LLMs learn to predict sequences of tokens, which can represent words, parts of words, punctuation, or pieces of code. This makes them great at automation tasks involving language, like coding and writing. However, LLMs have limits because the world is not made up of text. Now, an older idea in AI is gaining new momentum: world models. World models learn patterns in how an environment changes over time, including how it might change in response to an action. If you drop a glass, it falls. If you kick a ball, it rolls. If you open your mouth while it’s full of water, the water spills onto the floor. The goal is to learn enough about those patterns to predict what might happen next. They can learn in different ways. Some systems learn through interaction or simulated environments. Others learn by obse