Two Big AI Risks Revealed In Anthropic's New S-1 Filing

0:00When it's time to scale your business, it's time for Shopify. Get everything you need to grow the way you want. Like, all the way. Stack more sales with the best converting checkout on the planet. Track your cha-chings from every channel, right in one spot. And turn real-time reporting into big-time opportunities. Take your business to a whole new level. Switch to Shopify. Start your free trial today. This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome? That's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks.

0:44Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required compatibility and availability varies 18+. Today on Forbes, Two big AI risks revealed in Anthropic's new S1 filing. Anthropic officially filed for an initial public offering, or IPO, on Monday. That step brings it closer to what could be one of the largest public debuts in stock market history. The company is slated to go public in the coming weeks. Anthropik's S1 filing on Monday was the first glimpse public investors have gotten into the AI firm's financials.

1:28It also revealed numerous previously unknown aspects of how the company functions and how it views AI's role in society. In the filing, it revealed that more than 90 % of the company's code was written by its own AI tool, Claude, in September. The AI giant is increasingly using its own AI models to research, design, and develop subsequent versions of its models, a process it calls, quote, closing the loop. In the S1, the company wrote, quote, The company says that Claude runs experiments, generates training data, and automates evaluations and safety audits. It has also helped make algorithmic improvements that allow Anthropic to use computing resources more efficiently.

2:20According to the filing, Anthropic reported$16.3 billion in revenue and$693 million in operating losses in the first six months of 2026. In Q2-2026, its first profitable quarter, it had$974 million in operating income. It's not surprising that Anthropik's 5 ,300-strong staff relies heavily on Claude, but the company now spends more on internal AI usage than it does on salaries. According to the S1, the value of Anthropik's internal Claude usage at list prices has grown over 250 times from January 2026 to September 2026. Thanks to Claude, engineers contributed over 45 times more code per day in Q3 2026 than in 2024.

3:13But while Anthropic lists Claude's ability to improve itself as one of its most, quote, significant structural advantages, it also poses a major risk. As Claude helps build future versions of itself, it compresses timelines and accelerates the pace of AI development. Antropic says in the S1 that a key risk is AI moving so fast that it could outpace the company's ability to develop corresponding safety and security measures. Flaws or errors present in current versions of the model could pass down into later ones, making them harder to detect and fix. Antropic isn't the only company using its own AI to train more AI.

3:55Its rivals are doing it too. To address these risks, Anthropic is calling for an industry-wide initiative to, quote, pace the frontier and allow for independent safety reviews of all frontier models. Anthropic expects to spend$1 billion over the next five years to build, quote, external reviewer capacity with Accenture. Meanwhile, Anthropic also trumpets its commitment to using AI for scientific research on the very first page of its IPO prospectus. The introduction reads, We have started to directly make biological discoveries with Claude. It notes that Anthropic expects the model to make more breakthroughs in the future.

4:37It's referring to the AI firm's own internal biology research, announced last month, where its software discovered a new enzyme family with properties similar to those used for CRISPR gene editing. The company has also built a number of products for scientists, including Claude Science and also Claude for Life Sciences, which aims to help in drug discovery research, as well as helping to produce FDA submissions and similar paperwork. Andanthropic has developed software that enables AI agents to operate lab equipment, quote, such as microscopes, liquid handlers, and off-the-shelf robotic arms, to help automate research, according to the filing.

5:18It name-checks customers like pharmaceutical giant Bristol-Myers Squibb, the Garvin Institute of Medical Research, and the University of California, San Francisco. It earlier made headlines partnering with companies like Lilly and Novo Nordisk. Along with the potential rewards of applying Anthropics machine learning models to research come significant risks, especially in the realm of scientific research. That includes its model's ability to operate robots and lab equipment, which it says, quote, could result in bodily injury, property damage, damage to costly equipment or experimental materials, or the release of hazardous substances.

5:59This could be due to misalignments in the models, or deliberate misuse. Additionally, the company warns that its models could be useful to people, quote, seeking to develop chemical, biological, or other dangerous weapons. While Anthropic says it has deployed technical safeguards against bad actors who want to make weapons, it warns that, quote,

6:33For full coverage, check out Rashi Srivastava's and Alex Knapp's pieces on Forbes.com. This is Kieran Meadows from Forbes. Thanks for tuning in.

7:11Your stylist learns your size, your style, and your budget, then sends a box pick just for you. They always nail the fit. No store, no hassle, no subscription required. Get started today at StitchFix.com. That's StitchFix.com.