Presenters
Source
The AI Revolution: From Theory to Real Value and the Infrastructure Race 🚀
The world of technology is moving at breakneck speed, and at the heart of this progress lies the incredible evolution of Artificial Intelligence. We’re witnessing a paradigm shift, moving from theoretical AI concepts to tangible, real-world value, and this transformation is profoundly impacting the infrastructure that powers it all.
From AI Anticipation to AI Empowerment: A Journey 🗺️
For years, the focus was on building for AI, meticulously crafting infrastructure in anticipation of the immense value AI promised to unlock. We were optimizing for speed, flexibility, and innovation, driven by the unknown potential of AI products and their underlying infrastructure needs.
A year ago, the conversation centered on how AI was influencing systems innovation and vice-versa, leading to changes across the entire infrastructure stack. The key was optionality – preparing for an AI-driven future without knowing the exact shape it would take.
Today, we’ve transitioned from theory to real value from AI. The watershed moment? The emergence of agents. Unlike the single-session, search-like interactions of the past, agents unlock the ability to perform meaningful, end-to-end work on our behalf. They possess persistent memory, enabling more powerful and complex tasks.
The Agentic Revolution: Coding and Beyond 💻
Agents are now actively coding, not just for new applications but also for intricate and complex changes within decades-old codebases. At Meta, over 90% of core infrastructure changes submitted daily now contain AI-authored code. This is a testament to the tangible impact agents are having on productivity.
The value generated by AI is also flowing directly back into AI infrastructure itself. The announced CapEx spend by the five largest hyperscalers alone reaches a staggering $700 billion, a conservative estimate highlighting the unprecedented investment in this domain.
This moment mirrors the mobile internet revolution. Just as the internet in our pockets unlocked myriad new use cases, agentic execution is opening up AI in entirely new ways. Imagine a future where devices like Meta’s Ray-Bans, already capable of audio and photography, can seamlessly interact with your agent, allowing you to delegate tasks while remaining immersed in your daily activities.
The Infrastructure Race: Building for AI’s Demands 🏗️
The unprecedented growth in AI has ignited an intense infrastructure race. The supply landscape is a clear indicator:
- Scarcity: There simply isn’t enough of critical resources like data center power, network fiber capacity, fab capacity, silicon (GPUs and CPUs), and memory (HBM and DRAM) to meet current demand. Micron’s recent earnings announcement underscores this reality.
- Innovation at the Physical Layer: Companies are innovating at an unprecedented scale. Meta is unlocking up to 6.6 gigawatts of nuclear power in the US by 2035. Industry peers are exploring orbital data centers, signaling a move towards physical infrastructure solutions we could only dream of a few years ago.
- Co-design: A significant trend is the co-design of software and physical infrastructure, a theme that will be explored in more detail.
Meta’s Hybrid Cloud Journey: Embracing Heterogeneity ☁️
To secure necessary capacity, Meta has embraced a multi-cloud strategy, moving from solely owned data centers to running hundreds of megawatts of production workloads across multiple clouds. This journey, detailed in a dedicated talk on building the world’s largest hybrid cloud, has involved significant engineering effort.
- Twine Shared Layer: This host management layer now operates across Meta’s on-premises infrastructure, public clouds, and “neo clouds.”
- Heterogeneous Compute: Workloads are running on a diverse range of hardware, including H100s, B200s, GB200s, GB300s, Meta’s MTIA silicon, AMD GPUs (MI355Xs), and soon, Google TPUs.
- Storage Abstraction: To manage data movement challenges, Meta developed a “storage hub” allowing public cloud workloads to access storage in Meta’s data centers. Additionally, a lighter dependency stack, “storage light,” runs on the Twine shared layer.
- Performance Optimization: The team has scaled from simpler workloads like synthetic data generation to running reinforcement learning and model serving at scale on public clouds.
Rethinking Scale: The “Rings” Abstraction 🌐
The expansion into multiple cloud regions introduced a new scaling problem: the overhead of managing numerous geographically distributed services. To address this, Meta introduced “rings” – high-bandwidth network connections linking geographically proximate regions, regardless of their cloud provider. This abstraction allows services to communicate across regions seamlessly, enhancing flexibility and efficiency. Two rings are currently deployed, with plans for many more globally.
Building with AI: Enterprise Value and Safety 🤝
The focus now shifts to leveraging agents for meaningful work within the enterprise. This journey, initiated 6-9 months ago at Meta, presented unique challenges:
- Tool and System Interoperability: The sheer number of disparate tools and systems within an enterprise makes it difficult for agents to complete even simple tasks like scheduling meetings.
- Meta CLI: This foundational piece of technology acts as a directory service, enabling over 500 tools and systems to be discovered and called by agents within months.
- Meta IAC (Infrastructure as Code): This technology ensures that agents performing production operations on critical resources do so safely. It provides a robust framework for controlling agent actions on sensitive infrastructure, preventing unintended disruptions.
The “Below the Line” Foundation: Enabling AI Productivity and Safety 🛡️
Unlocking the true power of agents requires focusing on what lies “below the line” – the foundational work that makes AI itself productive and, critically, safe.
- Agent Safety: This is a paramount area of innovation. Meta is leveraging its existing infrastructure for privacy, security, and reliability, enhancing it for agentic operations.
- Guardrails: Agents are being equipped with guardrails to prevent actions like deleting user data.
- Recoverability: Recognizing that agents will make mistakes, Meta is investing in recoverability mechanisms. This includes minimal deployments, automated safety health checks, and automatic rollbacks based on these checks to minimize impact on end-users.
- Self-Improving Workflows: Similar to industry trends, Meta is implementing systems to automatically file tasks when agents falter, submit diffs, and continuously evaluate progress. The aspiration is a self-improving system that enables increasingly meaningful end-to-end work via agents.
Exploration at Scale: The Future of AI’s Impact ✨
The most exciting frontier for AI lies in its potential to accelerate progress for all of humanity. AI is already speeding up research in physics, biology, and medical sciences. Innovations like agent-enabled Ray-Bans are creating new communication possibilities for individuals with visual or hearing impairments.
This era allows for the rapid experimentation of dozens of ideas within the same week.
- Infrastructure Must Adapt: Large-scale infrastructure needs to step aside to allow for rapid experimentation. Simultaneously, it must facilitate a swift transition from prototyping to production scale, reducing timelines from weeks and months to days or weeks.
- Embracing Open Source: Meta is returning to its roots by embracing open source to facilitate experimentation. The ability to download and run models like GLM 5.2 within Meta’s infrastructure is crucial.
- Kubernetes Adoption: Within weeks, Meta launched a production-ready Kubernetes service (MKS), building on the Twine shared infrastructure. MKS is CNCF compliant, ensuring compatibility with existing Kubernetes ecosystems.
- Database Innovations: Postgres is experiencing a resurgence, with disaggregated storage and compute allowing for independent scaling. Agents can now spin up databases in seconds. Features like speculative execution, akin to source control for data, are being implemented.
- D SQLite: This new product offers full SQLite semantics with minimal operations, backed by Meta’s production-grade ZippyDB.
- Nest Platform: This Vercel-like application platform, built on function-as-a-service and D SQLite, has seen rapid adoption, with 22,000 internal applications built on it, including Meta’s internal chatbot, MetaMate.
- Project Hub: Meta’s backend services are now exposed as HTTP endpoints, allowing for easier integration with external projects via GRPC. The entire Meta AI product infrastructure layer utilizes this.
- Secure Isolation and Persistent Context: For agents to perform meaningful work, robust security and data privacy are essential. Meta has developed infrastructure for secure isolation and persistent context using Feather VM and storage volumes, built from scratch in just a few weeks.
The Infrastructure Stack: A Unified Vision 🧩
The infrastructure stack at scale is a complex interplay of layers:
- Heterogeneous Supply Layer: Encompassing data centers, public clouds, and neo clouds with diverse architectures.
- Foundational Software Layer: Abstracting away hardware differences, including K8s and S3-compatible storage APIs alongside Meta’s long-standing production infrastructure.
- AI Workloads Layer: The diverse applications and models running on top.
The key takeaway is the ability to co-design from the physical infrastructure layer all the way to the application layer.
The Road Ahead: Exponential Value, Exponential Work 📈
We’ve journeyed from the theoretical phase of building for AI to the early stages of realizing real value from AI. The next phase promises exponential value unlocked by AI, which, in turn, will drive exponential work in infrastructure.
This is our “AI moment,” akin to the railroads and electrification eras – periods of immense growth and potential in infrastructure. As a community, we possess the power to tackle these complex challenges. While the work is demanding, it is also an incredibly exciting time to be in infrastructure, pushing the boundaries of what’s possible.