Presenters

Source

Navigating the AI Wave: A Platform Engineering Survival Guide 🌊👨‍💻

The AI revolution is here, and it’s not just a fleeting trend – it’s a fundamental shift reshaping how we build and deliver software. For platform engineering teams, this means a new era of challenges and opportunities. While the hype is undeniable, the real impact of AI on our workflows demands a strategic, data-driven approach. Let’s dive into how platform teams can not only survive but thrive in this AI-powered future.

AI’s Double-Edged Sword: Velocity vs. Volatility ⚔️📊

Budgets for AI tools have doubled in the last year alone, signaling a strong trajectory. Early industry reports, like the Dora report, suggest modest positive gains in areas like documentation quality (a 7% increase) and code maintainability (a 3% increase). While these are good signs, they can be deceiving.

  • The Deception of Averages: When we break down these averages by company, a more complex picture emerges. Take the “change failure rate” metric. Some companies see a 50% increase in defects shipped after adopting AI, while others see a decrease. This volatility means we can’t blindly trust industry benchmarks. As platform teams, we must aim for the top line of this graph, minimizing those increased risks.

The New Mandate for Platform Engineering: Measurement is Key 🎯📈

So, what’s the new game plan for platform engineering teams in the age of AI? It boils down to a few critical pillars:

  • Data-Driven Rollout & Optimization: We need to move beyond just activity and focus on impact.
    • Measure Everything: Utilize existing productivity measurements, like the “core 4” metrics, to track what truly matters.
    • Correlate Utilization to Impact: Daily and weekly active users are important, but we need to link that usage to tangible improvements in foundational productivity metrics.
  • Build Paved Paths & Guardrails 🛣️🛡️:
    • Trustworthy AI Output: Engineers need to trust the AI’s suggestions, and platform teams need to trust that the AI is being used effectively and safely.
    • Bridge AI and Non-AI Workloads: Seamless integration is crucial for a unified developer experience.
  • Identify Real Bottlenecks, Not Just Hype 💡:
    • Beyond Code Generation: While AI can save a few hours a week on coding, the real friction often lies elsewhere. Think about heavy meeting days, context switching, and other sources of developer toil.
    • SDLC Integration: Focus on integrating AI to alleviate these actual pain points throughout the Software Development Life Cycle (SDLC).
  • Focus on the Bigger Picture: Developer Experience & Productivity ✨🚀:
    • Despite all the AI buzz, the ultimate goal remains building a better developer experience and enhancing overall productivity.
  • Control Infrastructure Costs 💰:
    • This is an ongoing challenge, even after 15 years of cloud adoption! We’re now seeing engineers burning through thousands of dollars worth of tokens daily. AI metrics will show tool usage, but foundational productivity metrics will reveal if initiatives are truly working.

A Framework for AI Measurement: The Core 4 & SPACE 🚀🛠️

To tackle this measurement challenge, a new AI measurement framework has emerged, built on the philosophy of “Core 4” and “SPACE.” This framework normalizes metrics into three key dimensions:

  • Utilization: Understanding what technology is being used and by whom. This is the starting point for most organizations.
  • Impact: This is where the magic happens. Are we actually achieving desired outcomes? This dimension focuses on foundational metrics like throughput and quality.
  • Cost: The final piece of the puzzle, ensuring our AI initiatives are financially sustainable.

Think of this as a maturity curve. We start by measuring utilization, then move to impact on productivity and developer experience, and finally, address cost.

Empowering Engineers with a Research-Driven Guide 📚👨‍💻

To help organizations navigate this new landscape, a research-driven guide to AI engineering has been published. This guide is designed for engineers and includes insights from:

  • Senior VPs: Sharing their strategies for establishing better ROI with AI tools, including practices like meta-prompting and multi-shot prompting, and creating effective feedback loops for system prompts.
  • Developers: Identifying their top five most valuable AI use cases, with many reporting saving at least an hour a week.

This guide is an excellent starting point for any organization looking to harness the power of AI effectively and responsibly.

The AI wave is here, and for platform engineering teams, it’s a call to action. By embracing a data-driven approach, focusing on real impact, and building robust frameworks for measurement and guidance, we can ensure that AI becomes a powerful ally in our quest for better developer experiences and more efficient software delivery.

Got questions or want to dive deeper into the AI measurement framework? Find us at the booth right outside the doors in front of the escalators, or join our office hours session at 10:30 AM in room B316!

Appendix