Anthropic Says It Is Working on AI Safety & Its Researchers Say the Problem Is Unsolved

• Anthropic AI Safety remains a major focus as AI Safety, Anthropic researchers, Superintelligent AI, and AI extinction risk concerns continue to shape the debate
• Anthropic researchers have raised concerns about developing increasingly capable AI systems before fully solving the safety and alignment challenges surrounding them
• Evan Hubinger said Anthropic has not yet solved alignment for superintelligent AI, while Jacob Coxon’s resignation brought renewed attention to the risks of self-improving systems

Anthropic AI safety concerns are now raising alarms as the company continues to face a new hurdle. The company has repeatedly assured the world how it follows the AI safety concerns to the core, prioritizing it as they continue to experiment. But the recent streak of events tells otherwise. An Anthropic researcher has now left the company, while other Anthropic researchers have publicly raised concerns about the risks associated with future AI systems. This sentence appears to carry a deeper meaning. The recent controversy has stirred due to Jacob Coxon, a researcher who worked at both OpenAI and Anthropic. Coxon left the company based on interesting concerns. His major concern was that Frontier AI companies are moving towards self-improving systems that may jeopardize humanity at large. He further stated how the companies are moving ahead with super intelligent AI without having fully resolved the safety and control questions surrounding such systems.

Also Read: The Rise of Violent Crypto Crime Is Turning Bitcoin Wealth Into a Liability

Anthropic Researchers Are Raising Constant Alarms

AI safety concerns are a serious issue that needs primary attention. The shift began the moment Jacob Coxon announced his resignation, citing concerns about the risks associated with advanced AI. His concern was backed by Evan Hubinger, Anthropic’s alignment science lead, who shared how he “earnestly believes AI could kill all humans.” Hubinger personally believes there is a greater than 10% chance that AI could kill all humans within the next decade.

Hubinger was also quick to highlight the limitations within the Anthropic AI safety model. He shared how the company is trying to address the problem but does not yet have a plan to solve alignment for superintelligence and is not clearly on track to do so. It is to be noted that Hubinger was discussing the future potential of AI, particularly systems with superintelligent AI capabilities. He further pointed out how such models can put humanity at risk if left uncontrolled or unsupervised.

What Is the AI Safety Problem in a Broader Sense?

The AI safety problem refers to efforts to make AI systems reliable, controllable and less likely to cause serious harm. AI alignment focuses on ensuring AI systems and their objectives remain aligned with human intentions. This may become difficult as companies continue to project building super-intelligent AI models in the future. Anthropic researchers have consistently raised alarms about such models, stressing the need to take AI safety concerns seriously and prevent future risk

Researchers are raising concerns about the risk to humanity from advanced AI, particularly as they examine how increasingly capable systems could behave without adequate safety and control mechanisms

Why Jacob Coxon’s Resignation Matters

Coxon’s comments highlight one of the central issues being debated in the AI world. His core argument focuses on Anthropic AI safety concerns. He said companies like OpenAI and Anthropic are competing to build better models, while he worries that increasingly capable systems could advance faster than the safety work needed to control them.

He does not claim that AI will definitely put humanity at risk. Instead, he warns that super-intelligent AI could pose a risk to humanity without adequate safety and control measures

Also Read: Gold Price Nears $4,400 as China Buys Gold, but Is It Dumping the US Dollar?

Anthropic’s Safety Position Meets the IPO Timing

This debate is unfolding at a critical time for Anthropic. Anthropic AI safety concerns are emerging as the company prepares for an IPO. Reuters reported that Anthropic is preparing for a potential IPO that may value the company at around $2 trillion
This figure is not a confirmed valuation but an estimated figure. At the same time, the potential valuation reflects the scale of expectations surrounding Anthropic and the broader AI market. Reuters earlier shared how investors are assessing the company against projected revenue of around $190B to $200B in 2028.

This development also creates a contrast, adding more weight to the Anthropic AI safety debate. The company is busy building new models and AI systems. There has been no evidence that states that the company is being irresponsible in any way. But this narrative shows that Anthropic researchers recognize the need for serious work on AI safety, particularly as they prepare for future superintelligent systems

Is AI Extinction Risk a Current Threat?

The Anthropic AI researchers’ comments should not be interpreted as a claim that current AI systems may cause human extinction. However, AI extinction risk is being seriously discussed by researchers working directly on advanced AI systems.

Anthropic’s own alignment science lead has publicly acknowledged that the company is yet to develop a plan to solve alignment for superintelligence. This should be the most important key takeaway from this entire narrative.

The real question behind this should be focused on creating AI systems that do not harm humanity. As the companies continue to compete against each other, intending to build super-intelligent AIs. The discussion around Anthropic AI safety concerns shows why safety and control remain an important part of the development of such models.

Also Read: Liquid Network Lost $320M Without a Compromised Key: What Does That Mean?