VGMG Global Report Series · AI Governance
【Lead】
After two years of informal dialogues and voluntary commitments, a critical mass of states is now pushing for a legally binding international treaty on advanced artificial intelligence (AI) systems. The impetus comes from recent demonstrations of autonomous reasoning capabilities in general-purpose models and growing concerns about military applications.
【Negotiating Landscape】
The “Geneva Track,” initiated in late 2025, has produced a draft framework requiring states to implement domestic safeguards for frontier AI models, including mandatory red-teaming, disclosure of training compute thresholds, and incident reporting.
A senior diplomat: “The divide is no longer between those who want regulation and those who don’t. Everyone agrees on the need. The fights are over verification, export controls, and exceptions for national security.”
The draft includes provisions for a new International AI Safety Agency (IAISA) under a UN mandate, modeled on the IAEA.
【Military AI – The Red Line Issue】
The most sensitive area remains the use of AI in weapon systems. The ICRC and UN Secretary-General reiterated that “meaningful human control must be retained over any weapon that can cause death or injury.” Leaked P5 discussion papers suggest a possible compromise—a ban on “unsafe autonomous targeting” without an outright prohibition on all autonomous defensive systems.
【Civilian Applications and Systemic Risks】
Negotiators are focused on systemic risks—the potential for frontier models to be used in cyberattacks, disinformation campaigns at scale, or to acquire dangerous biological knowledge. A coalition of AI labs has proposed a tiered approach—open-weight models below a certain capability threshold would remain unrestricted.
【Implementation and Verification Hurdles】
Unlike nuclear materials, AI models are purely informational and can be copied, transferred, or run in secret. The draft IAISA would rely on national reporting, on-site inspections at compute clusters, and third-party audits of cloud providers.
【Forward Outlook】
A high-level ministerial meeting scheduled for September 2026 in New York will attempt to finalize the text. If successful, the treaty could open for signature in early 2027.