Presenters

Source

From Moonshots to AI Missions: The Explosive Growth of AI Infrastructure 🚀

We’re living in an era of unprecedented technological advancement, and at the forefront of this revolution is Artificial Intelligence. But what does it really take to power this AI boom? Peter Hoose, who leads Production Engineering at Meta, dives deep into the colossal investments, engineering marvels, and the future of AI infrastructure, drawing parallels to the iconic moon landing.

The Astronomical Costs of AI 💰

Let’s start with some staggering numbers that put the scale of AI development into perspective:

  • $338 billion: This was the estimated cost over a decade to put a person on the moon. An incredible feat of human ingenuity and engineering! 🌕
  • $450 billion: This is the amount spent in 2025 alone on AI buildout. To give you a sense of scale, that’s roughly the GDP of Hong Kong! 🇭🇰
  • $765 billion: This is the estimated spend on AI infrastructure in 2026. That’s twice the cost of the Apollo and Gemini missions combined, all within a single year! 🤯

This massive investment is fueling rapid advancements, transforming how we work and interact with technology.

The AI Revolution in Action: From Autocomplete to Agents 🤖

Just over a year ago, AI coding assistants were rudimentary, offering little more than autocomplete. Fast forward to today, and tools like Claude Code and Codex are integral to daily development.

  • Seismic Shift: The release of Opus 4.5 around October marked a dramatic shift from simple code completion to agentic-driven development.
  • Daily Use: These AI tools now assist with everything from vacation planning and research to writing and complex code generation. They’re even building our infrastructure! 🛠️

This rapid evolution highlights the accelerating pace of AI development.

Building the Foundation: Data Centers and Hybrid Clouds ☁️

Peter Hoose takes us on a journey from the ground up, exploring the critical infrastructure enabling this AI surge.

Data Centers: From Theory to Reality 🏗️

  • Intents: What was once a theoretical concept with a few servers has become a critical part of Meta’s strategy.
  • Speed and Efficiency: Investments in “intents” allow for the utilization of land, power, and network resources in months rather than years.
  • Unforeseen Reliability: Rigorous engineering, including DR planning and fault injection, led to reliability far exceeding expectations. This involved electrical, mechanical, software, and systems engineering working in harmony.
  • Serving Capacity: This engineering effort now powers hundreds of megawatts of production capacity daily.

Hybrid Clouds: Scaling at Lightning Speed ⚡

  • Massive Investment: Meta has built one of the world’s largest hybrid clouds, investing billions with partners and scaling from a small cluster to tens of billions of dollars in just one year.
  • Rapid Deployment: This capacity can be brought online to serve production traffic in mere days, a stark contrast to the months or years it previously took.
  • Unexpected Challenges: The transition to the cloud revealed unexpected hurdles, particularly with cross-sectional bandwidth and heterogeneous hardware platforms. Systems optimized for consistent, high bandwidth in homogeneous environments required significant redesign to accommodate varying bandwidths and unique provider-specific hardware, software, and firmware.
  • Prometheus: The launch of Prometheus, a multi-gigawatt cluster, is now training some of the largest models in the world.

Abstractions: The Key to Seamless Integration 🔑

A crucial element of this infrastructure strategy is abstractions. Whether capacity resides in a data center, a cloud, or a multi-gigawatt facility, it’s abstracted away from the end user. This means the underlying hardware, whether it’s GPUs from NVIDIA, AMD, custom silicon like MTIA, or CPUs from AMD, ARM, or Intel, becomes invisible. To the products and services, it simply appears as “compute.”

  • Massive GPU Fleet: This integrated approach has resulted in over 1.3 million H100 equivalent GPUs at the start of 2026, with this number growing daily.

The Compute Layer: Redefining Reliability for Large-Scale Training 🧠

The evolution of compute, particularly with powerful GPUs and complex interconnects, demands a new approach to reliability.

  • GB200 Rack (Catalina): This rack, housing 72 GPUs and 72 CPUs, showcases advanced technologies like NVLink with thousands of custom copper interconnects.
  • The Interdependence Challenge: Traditional systems were designed to fail gracefully, with individual component failures having isolated impacts. However, large-scale AI training involves highly interconnected systems where the failure of one component can bring down an entire job.
  • Network Topology Model: The software has been redesigned to treat these systems as a large, interconnected cluster with complex interdependencies, resembling a network topology rather than a traditional distributed systems topology.

The Cost of Downtime: Compounding Failures 📉

The impact of job interruptions in large-scale training is significant:

  • Scalability Issues: As jobs scale from hundreds to tens of thousands of GPUs, individual failures compound, drastically increasing interruption rates.
  • GPU Idle Time: Without optimizations, large-scale training can lead to substantial idle GPU capacity. For example, with 100,000 GPUs, over 30% of capacity (around 40,000 GPUs) could be idle, representing a CAPEX loss of nearly a quarter billion dollars.
  • The Race Against Time: In a competitive AI landscape, every moment a job is down is time lost.

Engineering for Resilience: From Component to Cluster 🛠️

Meta has focused on reducing failure frequency and recovery time by:

  • Rethinking from Basics: Addressing failures at the component level (memory, CPU, GPU, control, IO).
  • Partner Collaboration: Working closely with partners to stress-test software and firmware, and improve diagnostics.
  • Intelligent Fleet Management: Developing systems that can identify server issues without impacting running jobs, allowing for repairs or replacements without interrupting critical training processes.
  • Dramatic Improvement: These optimizations have led to an estimated 95% reduction in failures and job interruptions.

The Software Revolution: AI-Powered Development and its Implications 💻

The shift towards AI-driven development is profoundly impacting software engineering.

  • Agents Writing Code: A significant portion of code in systems today is generated by AI agents, leading to a massive increase in developer productivity, with potential for 8x, 10x, and even 100x improvements.
  • Safer Changes, More Volume: Counterintuitively, AI-driven changes are safer. Over the past year, there’s been a 50% reduction in incidents per given change.
  • The Double-Edged Sword: While individual changes are safer, the volume of changes (diffs) has increased dramatically. This has led to an overall increase in incidents, despite improved safety per change.
  • GitHub’s Experience: GitHub has seen a need for 30x capacity increases to keep up with the volume of AI-driven development, far exceeding their initial 10x estimate.
  • Increased Toil: The time developers spend on operational tasks (“toil”) related to incidents and their remediation has increased by over 120% year-over-year. This underscores the importance of agent guardrails and agentic operations.

Real-World Impact: From Products to Vision Restoration ✨

The advancements in AI infrastructure are translating into tangible products and life-changing applications.

  • Agentic Behavior in Products: Users are experiencing agentic behavior and new AI capabilities in Meta’s products like Threads, WhatsApp, and Instagram.
  • Ray-Ban Meta: Giving Sight Again 👓: A particularly inspiring initiative is providing Ray-Ban Meta glasses to vision-impaired veterans. These glasses use cameras and microphones to capture the world around the user, sending data to inference systems that provide clear descriptions, effectively restoring their ability to “see.” This is a powerful testament to the end-to-end control from silicon to product.

The Sputnik Moment: The Dawn of a New Era 🌌

While the progress in the last year has been astounding, Peter Hoose emphasizes that we are still in the very early stages of the AI revolution.

  • Beyond the Moonshot: We are not at our “moon moment” yet. The current phase is better analogized as our “Sputnik moment” – the very beginning of a new space race.
  • Projected Investment: The estimated CapEx for AI in 2027 is a staggering $1 trillion. This figure is comparable to the entire budget of the U.S. military.
  • Future Challenges:
    • Fault Tolerance: Continued improvements in fault tolerance for training algorithms and detection/remediation systems are crucial.
    • Agentic Operations: Enhancing agentic operations is essential as code is only one part of the complex software lifecycle.
    • Hardware Constraints: The $1 trillion investment will exacerbate existing hardware constraints, leading to “everything constraints” (CPU, memory, disk) and increased complexity due to longer hardware lifecycles and greater heterogeneity.

The Human Element: The True Engine of Progress 🧑‍🤝‍🧑

Despite the immense technological leaps, Peter Hoose concludes by highlighting the indispensable role of people.

  • Control of Destiny: Having control over the entire stack, from silicon to product.
  • Speed and Autonomy: Reducing red tape and empowering individuals to move at unprecedented speeds.
  • The Power of People: The collective experience, talent, and collaboration of individuals are the driving force behind this progress.

This is an incredibly exciting time to be in the tech industry, working on some of the most challenging and impactful infrastructure build-outs in human history. The journey ahead is immense, but the potential for meaningful innovation and positively impacting lives is boundless.

Appendix