To determine whether a world model truly understands physics, you can’t just look at whether it can recite an explanation of gravity.

This featured clip from AI on Air is from a recent episode of Redpoint AI. The speaker is Oriol Vinyals, Google DeepMind’s VP of Research and a co-lead of Google Gemini.

▷ Core challenge: Once language is fed into a model, answers about “gravity” may come solely from explanations it read online during training. That doesn’t prove the concept is genuinely represented inside the world model.
▷ A practical evaluation: Connect whether “gravity” is present in the world model’s representations with explanations that can be decoded. First see whether it can provide a basic explanation, then whether it can derive equations.
▷ A methodological clue: Early unsupervised machine translation could align language representations for languages it had never encountered during training. Similarly, we can align a language model’s decoding abilities with a world model’s conceptual understanding—but the evaluation must ultimately be meaningful for applications.🪁