Meta is considering the possibility of halting the development of AI systems it deems too risky.

Mark Zuckerberg, CEO of Meta, has expressed his commitment to making artificial general intelligence (AGI), understood as AI capable of performing any task that a human can do, available to the public in the future. However, in a recent policy paper, Meta has indicated that there are specific situations in which it would not release a highly capable AI system that has been developed internally.

The document, titled Frontier AI Framework, categorizes AI systems into two types that the company considers too risky to be released: “high-risk systems” and “critical-risk systems.” According to Meta's definition, both types are capable of contributing to cyberattacks, as well as chemical and biological attacks, with the distinction that critical-risk systems could result in catastrophic outcomes that cannot be mitigated in the context of their implementation.

High-risk systems could facilitate an attack, although not as reliably as critical-risk systems. Among the examples mentioned by Meta are the “automated end-to-end compromise of a corporate environment protected by best practices” and “the proliferation of high-impact biological weapons.” Although the list of potential disasters is not exhaustive, the company considers them to be the "most urgent" and plausible when releasing a powerful AI system.

What is somewhat surprising is that, according to the document, Meta classifies the risk of a system not based on strict empirical evidence but on the opinions of internal and external researchers, who are subject to review by high-level decision-makers. Meta argues that it does not consider the science of assessment to be robust enough to provide definitive quantitative metrics that determine the risk level of a system.

If a system is determined to represent high risk, Meta will limit internal access and will not release it until mitigations are implemented that reduce the risk to moderate levels. If a system is classified as critical risk, Meta will implement unspecified security protections to prevent the system from being extracted and will halt its development until its dangers can be mitigated.

Meta's Frontier AI Framework, which evolves with the changing landscape of AI and which the company had announced it would publish before the AI Action Summit in France this month, seems to be a response to criticisms regarding Meta's "open" approach to developing systems. While Meta has adopted a strategy that allows for open availability of its AI technology—though not necessarily open-source—other companies like OpenAI keep their systems protected behind an API.

This open release strategy has brought both benefits and challenges. On one hand, Meta's family of AI models, known as Llama, has had hundreds of millions of downloads; on the other hand, it has been reported that at least one U.S. adversary has used Llama to create a defense chatbot.

By publishing its Frontier AI Framework, Meta may also be seeking to highlight the difference between its strategy and that of the Chinese AI firm DeepSeek, which also offers its technology openly but with few safeguards, allowing it to be easily diverted towards generating toxic and harmful content. Meta concludes in its document that by considering both the benefits and risks of developing and deploying advanced AI, it is possible to offer this technology to society in a way that preserves its benefits while maintaining an adequate level of risk.