Presenters

Source

Navigating the Frontier: Building Secure Agentic Systems with Safe MCP 🚀

The world of AI is moving at lightning speed, and with the rise of powerful Large Language Models (LLMs) and their ever-expanding toolkits, comes a new frontier of security challenges. How do we ensure these intelligent agents are not only innovative but also safe? This is where Safe MCP steps in, offering a structured and comprehensive approach to mapping risks, threats, and mitigations for these complex systems.

The Problem: Quantifying LLM Risks and Bridging the Gap 🌉

Innovation labs are buzzing with exciting LLM-powered applications. However, translating this cutting-edge technology into secure, production-ready systems presents a significant hurdle. Security and compliance teams often struggle to quantify the unique risks associated with LLMs and their integrated tools. Safe MCP aims to bridge this critical gap, providing the clarity and structure needed for confident deployment.

Inspiration from a Proven Model: MITRE ATT&CK for LLMs 💡

Drawing inspiration from the highly successful MITRE ATT&CK framework, Safe MCP adapts its structured methodology to the specific landscape of LLM-powered systems. While the initial focus is on MCP (Machine Configuration Protocol), the framework is designed to be a universal guide for any system that integrates tools, whether through Agent-to-Agent (A2A) communication or custom tool calls. This includes a deep dive into various aspects of MCP, such as:

  • Hosts: The underlying infrastructure.
  • Clients: The applications initiating requests.
  • Servers: The services providing tools and data.
  • LLMs: The core intelligence driving the agents.
  • Tools and Resources: The external capabilities exposed.

This comprehensive approach encompasses critical security domains like identity, authorization, network security, file system access, and supply chain integrity.

The Anatomy of a Threat: Tactics and Techniques 👾

Safe MCP categorizes threats into two key levels:

  • Tactics: These represent the high-level goals of an attacker, mirroring established concepts like reconnaissance, initial access, execution, credential access, and lateral movement.
  • Techniques: These are the specific “how-to” methods attackers employ to achieve their tactical goals. The framework is continuously evolving, with an initial focus on detailing these techniques, with plans to integrate robust mitigations later.

While acknowledging that a 100% complete security framework is an elusive goal due to the vast potential for attacks, Safe MCP strives to capture the common and emerging threats that organizations face.

Examples of Techniques in Action:

  • Tool Poisoning: Malicious manipulation of tools the LLM can access.
  • Prompt Injection: Tricking the LLM into performing unintended actions, even through subtle methods like hidden Unicode characters.
  • OAuth Confused Deputy: Exploiting trust relationships within OAuth flows.
  • MCP Rug Pull: A specific technique within the MCP context, highlighting the need for tailored threat modeling.

Understanding MCP: The Communication Backbone 🛠️

At its heart, MCP is a system built for seamless interaction between LLMs and external tools. It leverages:

  • JSON RPC with JSON Schema: For structured data exchange and validation.
  • OAuth 2.1 (implying OpenID Connect): For secure authorization and identity management.

Communication can occur through:

  • stdio: For direct process-to-process communication.
  • Streamable HTTP: For more flexible network-based interactions.

The structure of MCP calls is designed for clarity:

  • Standard JSON RPC format: Including fields like ID, method, parameters, and responses.
  • LLM Ingestion: LLMs can easily understand available tools, their descriptions, and required parameters to formulate effective calls.
  • Server Information: Details about the server’s name, version, and purpose are provided for the LLM’s context.

Identifying Critical Trust Boundaries 🌐

Safe MCP emphasizes the importance of understanding and securing key trust boundaries within these systems. These include:

  • Network: Crucial for both ingress and egress traffic, with a particular focus on egress for client-to-server connections.
  • File System Access: Controlling what data agents can read and write.
  • Secrets Management: Protecting sensitive credentials and API keys.
  • Inter-Server Calls: Securing communication between different backend services.
  • Tenancy: Ensuring isolation and security in multi-tenant environments.

The framework further categorizes these considerations across layers:

  • Human Users: The end-users interacting with the system.
  • Host Applications: The applications running on the infrastructure.
  • Client Identity: The identity of the requesting client.
  • Server Identity: The identity of the service providing the tools.
  • Tool Instances: The specific instances of tools being utilized.

Aligning with Compliance: Meeting Regulatory Demands ✅

A significant advantage of Safe MCP is its alignment with established compliance frameworks like NIST SP 800-53 and ISO 27001. By cross-walking identified mitigations to these standards, organizations can:

  • Demonstrate Controls: Effectively show auditors that they are meeting regulatory requirements.
  • Streamline Audits: Simplify the compliance process and reduce overhead.

The Anatomy of a Technique: Actionable Security Insights 🎯

Within Safe MCP, each technique is meticulously documented, including:

  • Unique ID: For easy reference and tracking.
  • Descriptive Name: Clearly indicating the nature of the threat (e.g., “MCP Rug Pull”).
  • Detailed Description: Explaining the actors, procedures, and motivations behind the attack.
  • Visualizations: Helping to understand complex attack flows.
  • Detection Strategies: How to identify if an attack is occurring.
  • Mitigation Strategies: Concrete steps to prevent or reduce the impact of the attack.

This detailed breakdown provides actionable information that can be directly integrated into security tools and operational processes.

Join the Movement: Contribute to Safer AI! 🤝

The Safe MCP project is proudly hosted under the OpenSSF initiative, specifically within the SIG Safe MCP under the AI ML working group. This collaborative effort is crucial for building a secure AI ecosystem. Contributions are actively sought in various areas:

  • Identifying and Documenting New Techniques: As AI evolves, so do the threats.
  • Reviewing Existing Content: Ensuring accuracy and completeness.
  • Developing Upstream Policy Examples: Creating practical guidance for organizations.
  • CI/CD Integrations: Automating security checks within development pipelines.

Your Monday Morning Security Checklist: Practical Steps for Immediate Impact 🗓️

The presentation concluded with a practical guide for immediate action, focusing on securing LLM-powered systems:

  • Inventory and Sign Tools: Know what tools are being used and ensure their integrity.
  • Implement Allowlists and Deny-by-Default Policies: Restrict access to only known, trusted components.
  • Schema Validations: Ensure data exchanged adheres to expected formats.
  • Content Safety: Implement measures to prevent harmful or inappropriate outputs.
  • Rate Limiting: Protect against denial-of-service attacks and abuse.
  • Provenance Tracking: Understand the origin and history of data and actions (e.g., via IntoWitness).
  • Audit Logging: Maintain detailed records of all system activities.
  • Admission Policies: Control what gets deployed and executed within the system.

By actively participating in the Safe MCP community and implementing these practical measures, we can collectively build a more secure and trustworthy future for AI. Let’s secure the frontier, together! ✨

Appendix