Google DeepMind has initiated a significant research effort aimed at understanding the potential risks associated with the interaction of millions of AI agents operating online. Rohin Shah, the director of AGI safety and alignment research at DeepMind, emphasizes that the widespread deployment of autonomous agents, which can perform tasks without human oversight and follow instructions from other agents, introduces a new class of risks. This concern has prompted Google DeepMind to collaborate with several organizations, including Schmidt Sciences and the UK’s ARIA agency, to establish a $10 million fund dedicated to exploring multi-agent systems and developing strategies to mitigate unsafe scenarios.
The objective of this funding is to spark research outside of traditional tech companies, as Shah notes that academia possesses the capability to investigate long-term implications that may not be prioritized within corporate labs. He acknowledges the current lack of a dedicated research field for multi-agent safety and expresses a desire for its establishment. As the deployment of AI agents increases, Shah warns that we could reach a critical juncture where speculative risks become tangible. He draws parallels to human institutions, which can achieve collective outcomes that no individual could manage alone, highlighting the need to understand the dynamic interactions between multiple AI agents.
The risks posed by these interactions include enhanced versions of existing online threats such as fraud and cyberattacks, prompting Shah and James Fox of Schmidt Sciences to advocate for rigorous simulation studies. By placing AI agents in controlled environments, researchers can observe their behavior in response to various stimuli. Fox points out that predictions cannot be made based solely on the behavior of individual agents; the complexity arises from numerous agents interacting simultaneously. Shah and Fox both underscore the urgency of addressing these risks before AI agents are integrated into economic systems in significant numbers. While other AI firms, including Anthropic, have also raised alarms regarding the safety of AI systems, Shah and Fox emphasize that understanding the new risks presented by agent-based technologies is critical for safeguarding digital infrastructure.
Source: Google DeepMind is worried about what happens when millions of agents start to interact via MIT Technology Review