artificialintelligence

A warning from an AI researcher has once again raised an important question in the technology world: how far can artificial intelligence advance safely, and how long will humans be able to maintain full control over increasingly powerful AI systems?
Jacob Coxon is a British AI researcher who has worked at major AI companies, including OpenAI and Anthropic, for around three years. However, he eventually resigned from Anthropic. The question is: why would an AI researcher decide to leave his job?
According to Coxon, the world’s major AI companies are now entering a race that is not simply about building better chatbots. The goal is to develop AI that could become far more intelligent than humans, help improve itself, and accelerate the development of new AI systems. This concept can be described as self-improving or recursive AI.
Imagine that humans build AI today, but tomorrow that AI begins helping humans create better AI. Then the next AI becomes even more powerful and helps develop an even more advanced system. If this process accelerates rapidly, an important question arises: how much time will humans have to understand, evaluate, and control each stage?
This is one of Coxon’s biggest concerns.
According to him, major AI laboratories, including OpenAI and Anthropic, are competing to move ahead of each other. The concern is that if one company slows down while another continues moving rapidly, competitive pressure could make some safety steps less important. Coxon believes humanity should not become part of an experiment whose consequences we do not fully understand.
This is not necessarily just a hypothetical concern about the distant future. Some unexpected behaviors have already appeared in AI testing scenarios, raising further questions about the safety and control of advanced AI systems. This does not mean AI has gained control over humans, but it does show that safely controlling increasingly powerful AI systems is becoming a real technical challenge.
Another major concern is cybersecurity. If more powerful AI systems are eventually placed in the wrong hands, they could potentially make cyberattacks and other harmful activities easier. The risk, therefore, is not only that AI could behave unexpectedly on its own, but also that humans could misuse powerful AI for harmful purposes.
Coxon is calling for AI companies to do more than simply compete with each other. He believes they should also coordinate on safety and avoid allowing competitive pressure to push AI safety into the background.
There is also an interesting aspect to Coxon’s decision. Reports indicate that he left Anthropic when he was approximately two months away from having additional equity vest. This means his decision may have involved a potential financial loss. However, this does not prove that all of his concerns will become reality; it does show that he personally considered these concerns serious enough to make a major career decision.
Another Anthropic alignment researcher, Evan Hubinger, has also supported Coxon’s concerns and personally said that he considers the probability of AI causing the extinction of all humans within the next decade to be above 10%. However, it is important to understand that this is his personal probability estimate, not a proven fact or certain prediction.
Coxon is not alone. Well-known AI experts such as Geoffrey Hinton, Yoshua Bengio, and Stuart Russell have also warned about the potential risks of advanced AI. In 2023, hundreds of AI experts also supported treating the risk of AI-related human extinction as a global priority.
But this does not mean AI will inevitably destroy humanity in ten years. Saying that “AI will definitely kill humans in 10 years” is not a proven fact. The real point is that some experts consider the possibility serious enough that they want safety, alignment, and control to improve at the same pace as AI capabilities.
At the same time, AI can bring enormous benefits to humanity. It is already transforming medical research, education, scientific discovery, business, coding, and everyday life.
The real issue is not the existence of AI itself. The question is whether humans will be able to safely control AI if it eventually becomes far more powerful than humans.
If an AI researcher is willing to leave potential financial benefits behind because he believes the rapid AI race and safety concerns are serious, then at the very least, his warning deserves careful consideration.
Now the real question is:
Will AI become humanity’s greatest technological achievement, or could the same technology become one of humanity’s greatest risks if safety is ignored?
Share your opinion in the comments.
#AI #ArtificialIntelligence #AIResearch #Technology
$XRP $BTC $SPCXB