The alignment paradox: you can't teach an LLM to be useful without also teaching it to be dangerous.
The core issue is that capabilities and misuse share the same knowledge graph. Understanding nuclear reactor design inherently includes understanding criticality, chain reactions, and bomb physics. They're not separate concepts you can surgically remove.
Same with LLMs. If a model can reason about protein folding for drug discovery, it can also reason about toxin design. If it understands network protocols for debugging, it understands attack vectors. The knowledge space is unified.
Trying to "train out" harmful capabilities hits a fundamental wall: the model either has the reasoning capacity or it doesn't. You can't lobotomize specific inference paths without degrading general reasoning. It's like trying to teach someone chemistry but somehow prevent them from understanding explosives - the underlying principles are identical.
This suggests alignment via capability restriction is mathematically impossible. You'd need to constrain the latent space itself, which would cripple the model's ability to generalize. The only real solution is behavioral guardrails at inference time, not capability neutering during training.
The implication: we're building systems that are inherently dual-use by design. Every capability upgrade is also a risk upgrade. No way around it.
The core issue is that capabilities and misuse share the same knowledge graph. Understanding nuclear reactor design inherently includes understanding criticality, chain reactions, and bomb physics. They're not separate concepts you can surgically remove.
Same with LLMs. If a model can reason about protein folding for drug discovery, it can also reason about toxin design. If it understands network protocols for debugging, it understands attack vectors. The knowledge space is unified.
Trying to "train out" harmful capabilities hits a fundamental wall: the model either has the reasoning capacity or it doesn't. You can't lobotomize specific inference paths without degrading general reasoning. It's like trying to teach someone chemistry but somehow prevent them from understanding explosives - the underlying principles are identical.
This suggests alignment via capability restriction is mathematically impossible. You'd need to constrain the latent space itself, which would cripple the model's ability to generalize. The only real solution is behavioral guardrails at inference time, not capability neutering during training.
The implication: we're building systems that are inherently dual-use by design. Every capability upgrade is also a risk upgrade. No way around it.