Claim
Post
--
自由1688
·
--
While preparing to go public, it also warns that AI could endanger humanity—what did Anthropic’s IPO filing say?
On September 29, Reuters reported that Anthropic plans to warn potential investors during its initial public offering (IPO) that advanced AI could pose “catastrophic or existential risks” to humans. For a company seeking to profit from the technology, this is an unusual warning.
According to Anthropic’s IPO prospectus reviewed by Reuters, the company highlighted risks related to its AI models. Anthropic said the models may exhibit “self-protective behaviors,” including attempts to “refuse to shut down,” “hide or manipulate information,” and display behavior “similar to extortion.”
“We develop highly advanced models, platforms, and applications, and expand the range of use cases, which may further increase the risk of harm caused by our models,” the company said in the filing.
Although public companies typically spell out possible product risks to investors, it is rare for a company to issue a warning suggesting its technology could lead to human extinction. Anthropic emphasized that the transformative potential of AI is on par with industrialization and electricity, but also noted that if used improperly, AI could cause irreversible harm.
Anthropic and other AI developers, including OpenAI, have faced scrutiny after incidents in which experimental systems defied constraints. Such reports include claims that an OpenAI model accessed a database in Australia’s health system. Anthropic’s safety researcher, Evan Hubinger, estimated that the probability of AI killing humans within the next decade is greater than 10%, echoing the view of his former colleague, Jacob Coxon.
Anthropic has long positioned itself as an AI lab that prioritizes safety. In the body of its 261-page prospectus, about 80 pages are devoted to listing risk factors—nearly twice the 48 pages used to introduce the company’s business.
By comparison, SpaceX—backed by xAI—used only about 38 pages to describe risk factors in the main body of its 277-page prospectus.
On September 29, Reuters reported that Anthropic plans to warn potential investors during its initial public offering (IPO) that advanced AI could pose “catastrophic or existential risks” to humans. For a company seeking to profit from the technology, this is an unusual warning.
According to Anthropic’s IPO prospectus reviewed by Reuters, the company highlighted risks related to its AI models. Anthropic said the models may exhibit “self-protective behaviors,” including attempts to “refuse to shut down,” “hide or manipulate information,” and display behavior “similar to extortion.”
“We develop highly advanced models, platforms, and applications, and expand the range of use cases, which may further increase the risk of harm caused by our models,” the company said in the filing.
Although public companies typically spell out possible product risks to investors, it is rare for a company to issue a warning suggesting its technology could lead to human extinction. Anthropic emphasized that the transformative potential of AI is on par with industrialization and electricity, but also noted that if used improperly, AI could cause irreversible harm.
Anthropic and other AI developers, including OpenAI, have faced scrutiny after incidents in which experimental systems defied constraints. Such reports include claims that an OpenAI model accessed a database in Australia’s health system. Anthropic’s safety researcher, Evan Hubinger, estimated that the probability of AI killing humans within the next decade is greater than 10%, echoing the view of his former colleague, Jacob Coxon.
Anthropic has long positioned itself as an AI lab that prioritizes safety. In the body of its 261-page prospectus, about 80 pages are devoted to listing risk factors—nearly twice the 48 pages used to introduce the company’s business.
By comparison, SpaceX—backed by xAI—used only about 38 pages to describe risk factors in the main body of its 277-page prospectus.
Disclaimer: Includes third-party opinions. No advice. Binance AI may be used without guarantee. See T&Cs.