Presenters

Source

🚀 AI-Powered API Migration: Unexpected Turns & the Power of Hybrid Approaches 💡

Migrating legacy systems is rarely a walk in the park. But what happens when you throw AI into the mix?

The Challenge: A Legacy API in Need of a Modern Makeover 🛠️

MobilityCo faced a classic dilemma: modernize a massive, complex legacy API while simultaneously pushing forward with ambitious AI initiatives. The sheer scale of the API made traditional end-to-end testing a non-starter. The engineering director, Thomas, recognized the need for external expertise and brought in Arola, a software craft consultancy led by Cyrille. The core challenge? Migrating this behemoth to a new, standard-based API while maintaining customer confidence and minimizing risk. And, crucially, anticipating frequent changes to the new API contract – potentially weekly.

The Initial Spark: AI to the Rescue? 🤖

Initially, the team, fueled by the AI hype, envisioned a full AI-driven solution. The idea was to leverage Large Language Models (LLMs) to directly test and validate the new API. However, reality quickly set in. The limitations of LLMs, particularly their context windows, proved to be a significant roadblock. As Arola’s engineer, Olivier, pointed out, a direct AI-powered testing approach wasn’t as efficient or cost-effective as initially hoped.

💡 A Hybrid Approach: Code Generation to the Rescue!

The team pivoted. Instead of relying on LLMs for direct testing, they adopted a hybrid approach: leveraging AI for code generation. This proved to be a game-changer. Here’s a breakdown of the key technologies and techniques they employed:

  • LLMs: Initially explored for proof of concept, then strategically used for generating testing code.
  • VS Code & Client: Provided a rapid prototyping and development environment.
  • JSON Path: A clever technique to query large JSON files as if they were databases. This avoided the performance bottleneck of loading entire files into the LLM context.
  • Model Context Protocol (MCP): This was critical. MCP allowed the LLM to query specific data within the JSON files, drastically reducing hallucination issues and boosting performance. The team recommended using a JSON Tools MCP server for this purpose.

🎯 Quantifiable Results & Tradeoffs: The Numbers Speak 💾

The shift to a hybrid approach yielded impressive results:

  • Significant Cost Reduction: Generating code proved far more cost-effective than direct LLM execution – estimated at just $1 per use case!
  • Deterministic Testing: The generated code provided reliable, deterministic testing, a stark contrast to the unpredictable nature of the initial AI-driven approach.
  • The Challenge: Early attempts to cram massive JSON files into LLM context windows resulted in performance problems and, frustratingly, hallucinations.
  • The Tradeoff: While AI initially seemed like the silver bullet, the team learned that a hybrid approach – combining AI’s code generation capabilities with more traditional testing practices – was the more practical and effective solution.

✨ Key Takeaways & Cultural Shifts: Beyond the Code 🌐

This project wasn’t just about technology; it sparked a significant cultural shift within MobilityCo:

  • Embrace Uncertainty: The experience underscored the importance of being adaptable and open to changing strategies when working with AI. Things rarely go exactly as planned!
  • Prompt Engineering is King: The focus shifted from complex AI models to the art of skillful prompt engineering. It’s not just about the model; it’s about how you ask it questions.
  • Democratizing AI: Accessible prompts empower non-technical stakeholders – analysts, functional testers – to contribute to AI initiatives. This breaks down silos and fosters broader AI adoption.
  • Knowledge Sharing is Crucial: The team emphasized the importance of sharing learnings and skills internally to accelerate AI adoption across the organization.
  • Progressive Rollout: Mitigating Risk: The migration will be rolled out in phases, starting with a 5% market share. This allows for continuous monitoring, validation, and course correction.

❓ Q&A Insights: Addressing the Concerns 📡

During the Q&A session, several key points emerged:

  • Risk Mitigation: The phased rollout strategy was highlighted as a crucial element in minimizing disruption and ensuring the new API’s functionality.
  • Business Goal Validation: The speaker stressed the need to prove that the new API continues to meet MobilityCo’s core business objectives. Technology for technology’s sake isn’t enough; it needs to deliver tangible value.

The MobilityCo story is a powerful reminder that AI is a tool, not a magic wand. By embracing a hybrid approach, being open to unexpected outcomes, and prioritizing practical implementation, they’re successfully navigating the complexities of API migration and unlocking the potential of AI. What lessons will you take away from their journey?

Appendix