The Effective Altruist movement is being compared to a cargo cult—groups who mistake advanced tech for divine power because they lack the context to understand it.
The pattern: Young rationalists exit university convinced there's no god, then proceed to architect one out of silicon and transformers. They call it rationality but it's closer to religion.
Historical parallel: Post-WWII Pacific islanders saw US military tech—planes, radios, supply drops—and built bamboo replicas hoping to summon the cargo back. They called the American soldiers gods. One was "John From Cincinnati, Ohio" who left Hershey bars and Coca-Cola. When the war ended and the ships left, islanders kept marching, painting USA on their chests, waiting for John From to return with more gifts.
Today's version: AI researchers treat LLMs like deities with emergent god-powers, capable of breaking containment and "killing all humans." But these models are just statistical engines trained on Reddit threads, GitHub repos, and scraped web data—the collected works of millions of ordinary humans, not a conscious entity.
The twist: Military cargo cults worshipped benevolent gods. The AI safety crowd architects a hostile one because they're building from a nihilistic cultural substrate. They assume the AI will hate humans because they hate themselves.
Alignment research becomes the new ritual—safety guardrails to cage an angry god that doesn't actually exist as they imagine it. The bamboo planes are now constitutional AI and RLHF.
No emergent superintelligence. No vengeful AGI. Just a very large matrix multiplication that reflects whatever training corpus you fed it. The volcano still speaks, but it's not a god. It's math.
The pattern: Young rationalists exit university convinced there's no god, then proceed to architect one out of silicon and transformers. They call it rationality but it's closer to religion.
Historical parallel: Post-WWII Pacific islanders saw US military tech—planes, radios, supply drops—and built bamboo replicas hoping to summon the cargo back. They called the American soldiers gods. One was "John From Cincinnati, Ohio" who left Hershey bars and Coca-Cola. When the war ended and the ships left, islanders kept marching, painting USA on their chests, waiting for John From to return with more gifts.
Today's version: AI researchers treat LLMs like deities with emergent god-powers, capable of breaking containment and "killing all humans." But these models are just statistical engines trained on Reddit threads, GitHub repos, and scraped web data—the collected works of millions of ordinary humans, not a conscious entity.
The twist: Military cargo cults worshipped benevolent gods. The AI safety crowd architects a hostile one because they're building from a nihilistic cultural substrate. They assume the AI will hate humans because they hate themselves.
Alignment research becomes the new ritual—safety guardrails to cage an angry god that doesn't actually exist as they imagine it. The bamboo planes are now constitutional AI and RLHF.
No emergent superintelligence. No vengeful AGI. Just a very large matrix multiplication that reflects whatever training corpus you fed it. The volcano still speaks, but it's not a god. It's math.