Scale Systems and Reliability 2026

Presenters Gaurav Metra Source Scale Systems and Reliability 2026 AI: Your New On-Call Partner for Unbreakable Systems 🚀 Remember that gut-wrenching feeling when a critical system goes down? Gaurav Metra, a Production Engineer at Meta, sure does. He vividly recounts a harrowing Wednesday morning on December 11th, 2024, when a single configuration change triggered a widespread outage, impacting all of Meta’s apps for a grueling four hours. This wasn’t just a technical hiccup; it was a “sev zero” incident, requiring hundreds of engineers across dozens of teams, leading to sleepless nights and lost roadmap progress. ...

Presenters Dinesh Govindasamy Sargun Dhillon Source Scale Systems and Reliability 2026 Meta’s Bold Leap to the Cloud: From On-Prem Masters to Hybrid Powerhouses ☁️🚀 Meta, the tech giant behind Facebook, Instagram, and WhatsApp, has long been a pioneer in building its own incredibly robust and efficient on-premises infrastructure. For years, they meticulously crafted every layer of their tech stack, from custom hardware in their data centers built through Open Compute to specialized software like HHVM for their social graph. This “code design” approach, where hardware and software are optimized together, fueled their rapid growth and reliability. ...

Presenters Phil Lopreiato Rahul Iyengar Source Scale Systems and Reliability 2026 When Meta Almost Had a Really Bad Day: A Deep Dive into a Near Catastrophe 🤯 Hey tech enthusiasts! Gather ‘round, because we’ve got a story that will make your palms sweat and your admiration for robust infrastructure soar. Phil Lopreiato and Rahul Iyengar from Meta’s Web Foundation team are here to pull back the curtain on a nail-biting incident that could have spiraled into a major outage, reminding us that even the giants of the tech world face hair-raising challenges. ...

Presenters Joe Romano David Pariag Sargun Dhillon Joe Duffy Source Scale Systems and Reliability 2026 Guardrails, Agents, and the Future of Infrastructure Management: A Deep Dive 🚀 The world of cloud infrastructure is evolving at lightning speed, and with the rise of autonomous agents, the need for robust guardrails has never been more critical. We sat down with some brilliant minds – Joe Romano, David Pariag, Sargun Dhillon, and Joe Duffy – to explore how we’re building safety, efficiency, and intelligence into the very fabric of our infrastructure management. ...

Presenters Phillip Liu Source Scale Systems and Reliability 2026 Taming the NCCL Watchdog Timeout: Meta’s Journey to AI Reliability 🚀 Ever hit a wall with your AI training jobs? Those dreaded “NCCL watchdog timeout” errors can feel like a black box, leaving you scratching your head and wasting precious compute time. Well, buckle up, because we’re diving deep into how Meta tackles this beast, making complex debugging accessible and even exciting! ✨ ...