Oxford researchers Teppo Felin and Matthias Holweg just dropped a paper that cuts through the AGI hype: LLMs fundamentally cannot invent anything new because they're architecturally backward-looking.
The core argument: LLMs are next-token predictors trained on existing data distributions. They can't hold beliefs that contradict their training corpus. Every major breakthrough in history—flight, relativity, quantum mechanics—started with someone believing something the data said was impossible.
The Wright Brothers example hits hard: In 1903, expert consensus and all available data said human flight was 1-10 million years away. Nine weeks later, they flew. They didn't have better data—they had a causal theory, built custom wind tunnels, decomposed the problem into lift/propulsion/control, and generated evidence that didn't exist yet.
This is the data-belief asymmetry: Humans theorize first, then run experiments to create new data. LLMs can only interpolate within the statistical envelope of what already exists. They're incredible at compression, remixing, and acceleration—but they can't independently discover new physics, energy sources, or medical paradigms.
The tools are powerful for what they do. Just don't confuse statistical pattern matching with the kind of causal reasoning that drives actual scientific revolutions. The next breakthrough still requires a human willing to say the existing record is wrong.
The core argument: LLMs are next-token predictors trained on existing data distributions. They can't hold beliefs that contradict their training corpus. Every major breakthrough in history—flight, relativity, quantum mechanics—started with someone believing something the data said was impossible.
The Wright Brothers example hits hard: In 1903, expert consensus and all available data said human flight was 1-10 million years away. Nine weeks later, they flew. They didn't have better data—they had a causal theory, built custom wind tunnels, decomposed the problem into lift/propulsion/control, and generated evidence that didn't exist yet.
This is the data-belief asymmetry: Humans theorize first, then run experiments to create new data. LLMs can only interpolate within the statistical envelope of what already exists. They're incredible at compression, remixing, and acceleration—but they can't independently discover new physics, energy sources, or medical paradigms.
The tools are powerful for what they do. Just don't confuse statistical pattern matching with the kind of causal reasoning that drives actual scientific revolutions. The next breakthrough still requires a human willing to say the existing record is wrong.
