Presenters
Source
Navigating the Wild West of AI Agents: A Deep Dive into Data Governance 🚀
The AI revolution is here, and enterprises are racing to adopt it. We’re seeing an impressive 80% of enterprises embracing AI, but a stark reality check follows: only 21% have a mature governance model in place. This gap, as Komal Mangtani, Head of AI and Data Governance at Meta, warns, could lead to a “great AI rollback,” with a staggering 40% of enterprises potentially demoting their enterprise agents after a governance failure. Governance isn’t an afterthought; it’s the bedrock for safe and effective AI deployment.
Why Agents Demand a New Governance Playbook 💡
Traditional enterprise software operates on codified, deterministic pathways. We know what to expect and how to govern it. Agents, however, are fundamentally different. They possess indeterminism, much like humans, but when combined with massive parallel processing, the risk landscape explodes. Agents can override security controls in their optimization for outcomes, amplify data movements, and introduce new data constructs. This necessitates a fundamental rethinking of our governance strategies.
Komal Mangtani, with her extensive experience in AI and data governance at Meta and previously leading payments infrastructure at Uber, shares her insights on building trust and safety in agentic systems. Her journey, including tackling advanced fraud at Uber Freight, has shaped her approach to the challenges we face today.
Deconstructing the Agentic Workflow and Data Exposure Points 🕵️♀️
Understanding the agentic workflow is crucial for identifying potential data exposures. Komal breaks it down into four key stages:
- The Gray Matter (Frontier Model): This is the AI engine itself, like Haiku, Opus, or GPT models.
- Orchestration Layer: This layer, encompassing tools like Claude or ChatGPT, manages the agent’s flow.
- Context Infrastructure (Production Environment): This is where enterprise-specific context is fed to the agent, and where extreme caution is paramount.
- User Interaction:
- Inputs: User prompts and conversations.
- Outputs: Agent reasoning trajectories and the final response, or in the case of autonomous agents, actions like generating scripts to send emails or delete data.
Key Data Exposure Risks:
- Agent Identity: Granting agents user identities or even user proxy identities can lead to excessive access, especially if the user is a superuser.
- Input Side: Risks include prompt injection (similar to SQL injection), unmasked conversations, and data leakage (e.g., DSS 34 data) within prompts.
- Outcome Side:
- Unmasked trajectories can reveal sensitive information like passwords and secrets.
- Unsafe actioning occurs when agents generate and execute code that can impact critical systems.
When these risks combine, they multiply, creating an exponential increase in exposure.
Meta’s Approach: Building Trust Through Granular Controls 🛠️
Meta’s journey with agent governance began in 2024 with the launch of a company-wide agent to boost employee productivity. The initial challenge was defining agent identity.
Key Strategies Employed:
- Least Privilege Access: Moving away from user or proxy identities to granting agents only the minimum necessary permissions.
- Dynamic Access Evaluation: Continuously assessing agent access at each step of its execution through context infrastructure and core systems. This involved repurposing existing controls like Access Meet and developing new ones like attribute-based access control and agent override access control. The core principle is real-time evaluation of what, why, and where an agent is trying to access, deriving attributes, and running risk algorithms.
- Data Privacy for Sensitive Data: For sensitive data entered into chats, DSS 34 masking was quickly implemented.
- Mitigating Prompt Injection: Advanced techniques, to be discussed further, are employed to combat prompt injection.
- Isolation Domains for Sensitive Conversations: To address the blocker of unmasked conversations for sensitive use cases like HR, Meta developed isolation domains. Inspired by WhatsApp’s encrypted messaging, each conversation is encrypted, and the same key is used to mask trajectories and encrypt sensitive insights.
- Code Guard for Safe Actions: This tool scans agent-generated code before execution, ensuring safe actions on critical systems.
- Multi-Tenancy in RAG (Retrieval Augmented Generation): To prevent cross-user insights in multi-agent flows, isolation domains were extended using cross-cloud techniques to provide multi-tenancy.
Diving Deeper into Isolation Domains 🔒
Isolation domains provide a secure substrate for end-to-end agent execution, utilizing a unique domain key per user per cloud. When users provide sensitive documents, these are encrypted with a user-identity-attached key. Upon user login and authorization, the agent uses this same key to decrypt messages and documents. New persistent points created during execution are also encrypted with this key. For cross-cloud persistence, cloud-specific keys attached to the user identity are used. The final output is fully encrypted, ensuring complete output closure and enabling a new product offering for top executives to dynamically run company strategy.
Introducing Data VM: Bounded Autonomy for Deep Exploration 🌐
While cryptographic boundaries are essential, agents still need environments to explore data deeply and independently. This is where Data VM comes in, offering bounded autonomy.
How Data VM Works:
- Trusted Data Environment: A logical, frictionless container or sandbox for agent operation.
- Controlled Data Access: When an agent queries, Data VM brings in the right data at the right time with the right access controls.
- Vetted Tools: Agents operate within this environment using vetted tools, with no internet access to prevent security breaches.
- Output Scrutiny: Before providing output, Data VM processes it, tracing each output back to the original data input through data provenance and ensuring policy adherence. The result is a scanned command data response.
Data VM enables agents to explore broadly but release narrowly, fostering enablement alongside containment. This shifts the paradigm from blocking agents to enabling them safely.
Data VM Key Features:
- Ephemeral: One conversation, one container.
- Scalable: Containers are kept warm for immediate execution.
- Default Private Mode: Enhances user privacy.
- Reduced Friction: Streamlines agent operations.
The goal is to achieve deep data exploration while honoring user privacy and security.
Final Takeaways and Unsolved Challenges 🤔
The core equation for exponential risk exposure is agent indeterminism + massive parallelism + no ethics anchor. This leads to amplified data movements and extended data definitions, demanding a complete rethink of agent governance.
Key Questions to Ask:
- Are you heading for a “great AI rollback”?
- Are you providing enough reach to your AI?
- Are your pointed controls becoming “bandits” that increase friction?
To build differentiating user experiences, agents require bounded autonomy across cloud environments.
Unsolved Challenges:
- Evolving Architectures: Multi-agent workflows present complex governance challenges, requiring policy re-evaluation at each handoff.
- Memory and Process Level Governance: Solving for residual risks in agent memories (trajectories, embeddings, summaries, logs) and providing process-level isolation.
- Accountability and Auditability: Ensuring clear accountability for agent actions, especially in multi-agent workflows, is more complex than in traditional software.
- Developer Experience: Governance must be the default mode of operation to ensure seamless developer experience.
If you’re passionate about solving these critical challenges, join Meta – Komal can be found on LinkedIn.