In the wake of increasing AI capabilities and risks, CEOs are facing pressing inquiries from their boards regarding the management of agentic systems. With recent guidance from regulatory bodies and technology providers emphasizing the need for robust security measures, a paradigm shift is in order: agents must be treated as powerful semi-autonomous users. This article outlines an actionable eight-step plan to enhance governance and security at the boundaries where agents interact with identity, tools, data, and outputs.
The first step in this framework involves constraining capabilities. Each agent should be treated as a non-human principal, much like an employee, with specific roles and permissions based on their function and geography. This means that agents must operate under clearly defined identities, prohibiting any shortcuts that could lead to unauthorized access. The principle of requiring explicit human approval for high-impact actions is crucial, aligning with guidance from frameworks like Google’s Secure AI Framework (SAIF) and NIST’s access-control guidelines. Additionally, it’s essential to control the tools agents can access, ensuring that toolchains are treated as secure supply chains, with strict versioning and approval processes governing any changes.
The latter stages of this plan focus on managing data and behavior. Organizations must view external inputs as potentially hostile and implement strict gatekeeping measures before any content enters the system. This involves reviewing and tagging new data sources to ensure they meet safety standards. Output handling is equally critical; any action based on AI-generated outputs should be validated before execution. Furthermore, data privacy must be prioritized, ensuring sensitive information is protected through tokenization and only revealed to authorized users under controlled circumstances. Finally, continuous evaluation is vital to maintain governance and resilience. Regularly testing agents against adversarial scenarios and maintaining an up-to-date inventory of models, tools, and datasets will empower organizations to swiftly adapt to emerging threats, thereby enhancing their overall security posture.
Source: From guardrails to governance: A CEO’s guide for securing agentic systems via MIT Technology Review