Nearly 40% of webpages from 2013 have vanished from the internet. This means modern AI models were trained predominantly on the surviving content - the stuff that was easy to monetize and archive.
This creates a massive selection bias in training data. AI models aren't learning from the full spectrum of human knowledge and culture from that era. They're learning from whatever content had commercial incentive to persist: corporate sites, ad-supported platforms, paywalled archives.
The implication? Current LLMs have a skewed worldview based on economically viable content, not necessarily the most accurate or diverse information. The internet's memory is selective, and AI inherited that selectivity.
This is why models sometimes hallucinate or lack nuance on topics that existed in now-dead forums, personal blogs, or non-commercial knowledge bases. The training corpus has survivorship bias baked in at the infrastructure level.
This creates a massive selection bias in training data. AI models aren't learning from the full spectrum of human knowledge and culture from that era. They're learning from whatever content had commercial incentive to persist: corporate sites, ad-supported platforms, paywalled archives.
The implication? Current LLMs have a skewed worldview based on economically viable content, not necessarily the most accurate or diverse information. The internet's memory is selective, and AI inherited that selectivity.
This is why models sometimes hallucinate or lack nuance on topics that existed in now-dead forums, personal blogs, or non-commercial knowledge bases. The training corpus has survivorship bias baked in at the infrastructure level.