Anthropic IPO Filing Warns AI Could Blackmail People, Become Sentient or End Humanity

Publication of the S-1 prompted discussion over whether Anthropic's quickly ballooning revenues will be able to overcome its even higher costs, according to Ars Technica. The filing also carries forward-looking optimism, saying AI is "rapidly transform[ing] every sector of the global economy in ways greater than the impact of industrialization, electricity, semiconductors, and the internet."

In the same document, Anthropic spends an inordinate amount of time laying out the myriad risk factors that could bring the company down, Ars Technica reported. The S-1's "Risk Factors" section runs across 93 pages and more than 70,000 words and discusses literally hundreds of potential problems that could negatively affect its business in the coming years.

Many of those risks are standard concerns almost any business must weigh, including competition, costs, customer acquisition and potential regulation. Others, Ars Technica said, highlight potential points of failure specific to Anthropic and to the broader hype-filled AI bubble. The article said it reviewed Anthropic's book-length treatment of its greatest risks and narrowed the list to the seven most surprising, interesting or thought-provoking potential problems, while describing that list as far from comprehensive and as something potential Anthropic investors should keep in mind.

The filing's warnings about AI blackmail, sentience and human extinction appeared in Ars Technica's account of the document. The S-1's publication came as investors weighed Anthropic's revenue growth against the costs of building and running advanced AI systems.