Presenters
Source
The AI Code Revolution: Why Infrastructure is the New Bottleneck 🚀
The year is 2026, and something incredible is happening in the world of software engineering. Engineers are writing more code than ever before. This surge in productivity, estimated to be anywhere from 2x to a staggering 10x improvement in shipping app code velocity, is largely thanks to the advancements in Large Language Models (LLMs). We’re even on a path to see 100x improvements with autonomous agent teams churning out application code around the clock.
But here’s the catch: infrastructure isn’t keeping pace. 🚧
While AI is making us more efficient at writing code, we’re understanding it less. This might sound counterintuitive, but it’s actually a good thing! We’re relinquishing more control to these increasingly capable agents, meaning humans have to toil away less. We can now ship the equivalent of multiple compilers per month. However, this disconnect between code generation and infrastructure readiness is creating new bottlenecks.
The “IKEA Furniture” Problem: Code is Easy, Deployment is Hard 🛋️
The core issue is that while AI excels at generating code, it struggles with the complex, often manual, process of deploying and managing that code in the real world. As Joe Duffy eloquently puts it, building a modern app is becoming like assembling IKEA furniture. You have all the pieces, but figuring out how to configure cloud services, manage API keys, handle dev vs. prod environments, and deal with secrets is a DIY nightmare.
This is where the new bottleneck emerges. AI can write the code at an unprecedented speed, but getting that code running where it needs to be has become the major hurdle. This involves:
- Manual Steps: Navigating cloud consoles, managing API keys, and copying/pasting YAML configurations.
- Lack of Agentic Aptitude: Agents aren’t naturally good at these complex, often unstructured, infrastructure tasks.
The reality is, infrastructure challenges like efficiency, scalability, cost control, security, and compliance don’t disappear just because AI is writing the code. In fact, with more capable AI models, we’re even seeing new security and attack vectors emerge.
The In-Distribution Advantage: Why Code is King for AI 👑
So, why is coding so much better suited for AI than infrastructure? It boils down to “in-distribution” versus “out-of-distribution” data.
- In-Distribution (Code): Billions of lines of code are publicly available on platforms like GitHub, along with extensive documentation in comments and markdown files. This vast, well-structured, and self-documenting dataset is precisely what LLMs thrive on. Developers also provide a strong feedback loop through verifiable code correctness.
- Quantification: The SWE-bench benchmark, a gold standard for AI coding capabilities, has seen an incredible jump from 33% accuracy less than two years ago to 86% with frontier models today. This represents a 75% reduction in errors! Anthropic’s Mythos model is already hitting 94%, and by the end of 2026, most frontier models are expected to achieve at least 95% on SWE-bench.
- Out-of-Distribution (Infrastructure): Infrastructure tasks like clicking through cloud consoles, managing API keys, or copying/pasting YAML are scattered, lack a standardized approach, and are rarely documented in a way that AI can easily learn from. There’s a vanishingly small training signal for these practices compared to general-purpose code.
This asymmetry means that while AI can naturally “code,” it struggles with the “infrastructure” aspects.
The Reframe: Turning Infrastructure into a Coding Problem 💡
The key to unlocking fully agentic infrastructure lies in a simple yet powerful reframe: express infrastructure problems as coding problems. Instead of trying to teach AI the spoken infrastructure practices, let’s leverage its fluency in code.
This is where Infrastructure as Code (IaC), particularly with a programmatic approach, becomes revolutionary.
Beyond Infrastructure as Text: The Power of Code ✍️
We’ve come a long way in managing infrastructure:
- Manual Era: Point-and-click in cloud consoles, manual CLI commands, and runbooks. Imperative, error-prone, and not repeatable.
- Configuration Era: Tools like Ansible, Terraform, and Salt Stack emerged, along with YAML for Kubernetes. While an improvement, it was largely “infrastructure as text” – more declarative but less expressive.
- Code Era (The Future): Using full programming languages (like Python, TypeScript, Go) to express infrastructure. This combines the declarative nature of IaC with the full expressiveness of code.
This approach, championed by tools like Pulumi, offers significant advantages:
- Ergonomic for Humans and Agents: What’s good for humans is also good for AI.
- Declarative and Auditable: You know what will happen before it happens, and it’s fully auditable.
- Guardrails for Autonomy: IaC provides essential guardrails, ensuring agents operate within defined boundaries.
Real-World Impact: From Training Clusters to Autonomous Teams 🌐
Imagine defining a complex training cluster for AI workloads using just Python code, as demonstrated with a Pulumi example:
- Defining Reusable Components: Encapsulating complex configurations like a CoreWeave Kubernetes cluster and an NVIDIA AI cluster runtime stack within a Python class.
- Dynamic Resource Allocation: Spinning up an RL stack with specific GPU types and node counts, using for loops to scale resources dynamically.
- Predictable Changes: Leveraging
git diff-like previews to understand the impact of infrastructure changes before deployment. This is crucial for both humans and agents to verify intent.
This “code all the way down” philosophy is already transforming how leading companies manage their infrastructure:
- Top Frontier Lab: Scaled training and inference infrastructure by 10x in a year using code.
- Wiz: Manages over 1 million resources and performs over 100,000 deployments per day.
- Superbase: Stands up new regions in less than a week.
- Composable AI: A startup with 100% agentically managed infrastructure, achieving more with a lean team.
Today, over 20% of infrastructure deployments are agentic, and the trend points towards 100% agentically managed infrastructure in the future.
The Path to Full Autonomy: Balancing Risk and Reward ⚖️
The road to full autonomy isn’t without its challenges. We can’t just unleash agents without consideration. However, we’re already seeing the benefits of autonomous workflows:
- Cleaning Cloud Waste: Agents can identify and remove unused resources, saving significant costs. The risk of an agent deleting something in a dev account is outweighed by the potential reward of saving thousands of dollars daily.
- Tagging Resources and Fixing Compliance: Proactively addressing untagged resources and compliance issues is becoming standard.
While some tasks, like deploying to production or setting up new projects, will remain human-assisted or supervised, the future points towards self-managing teams of agents handling infrastructure. This frees up humans to focus on higher-value, creative, and strategic activities.
By embracing the idea of infrastructure as a coding problem, we’re paving the way for a future of truly autonomous, efficient, and scalable infrastructure. The possibilities are immense, and the journey is just beginning! ✨