Presenters

Source

Supercharge Your AI Development: How Backstage Templates Are Revolutionizing Productivity 🚀

The world of Artificial Intelligence is moving at lightning speed. With the explosion of Large Language Models (LLMs) and sophisticated model servers, keeping up can feel like chasing a runaway train. For development teams, this means constantly navigating new releases, ensuring compatibility, and upholding stringent security standards. It’s a complex dance that can easily pull focus away from the core mission: building amazing AI applications.

But what if there was a way to streamline this entire process? What if you could equip your developers with a powerful toolkit that handles the heavy lifting of infrastructure and configuration, freeing them to innovate? That’s precisely the revolution Red Hat is spearheading with Backstage Software Templates, and it’s a game-changer for AI development.

The Magic of Predefined Blueprints 🎨

Think of Backstage Software Templates as your organization’s golden path for building new software. They’re standardized blueprints that automate the initial setup of any new software component, service, or application. This means every project kicks off with a robust, ready-to-go foundation, where best practices, security protocols, and testing frameworks are baked right into the code.

This “golden path” approach ensures absolute consistency across all your services. Crucially, it automatically aligns every project with your organizational compliance and security standards. This is especially vital when it comes to controlling the use of approved AI models, ensuring a secure and compliant AI ecosystem.

Abstracting Complexity, Unleashing Agility 🧘

The true power of these templates lies in their ability to abstract away the complexities of the ever-evolving AI landscape. By providing a standardized interface, developers automatically benefit from updates for compatible LLMs and model servers with minimal effort. This “plug and play” philosophy decouples your applications from specific models, empowering your teams to effortlessly evaluate, swap, or integrate new models with only minor code adjustments. The result? Unprecedented agility and a significant reduction in development friction.

The Architecture: A Modular Masterpiece 🏗️

The AI application architecture championed here is built on three elegantly modular components:

  • AI Application: This is the heart of your innovation – the core application logic. Frameworks like Streamlit are excellent for crafting user-friendly frontends, and you might even incorporate elements from valuable community projects like the “AI Recipes” from the MLOps community.
  • Pipeline: A centralized CI/CD pipeline, powered by technologies like Tekton Pipelines, orchestrates the automated building, testing, and deployment of your applications. This ensures a smooth and efficient release cycle.
  • GitOps: For managing deployments, tools like Argo CD Applications come into play. They leverage customized overlays to manage environment-specific deployments, ensuring consistency and control.

These components are designed for independence, with each residing in its own GitOps repository. This allows for independent development and version control. The magic happens with an “assembly line” process that combines these modules with a core template skeleton. This master file, guided by simple property files, compiles and customizes them into specific, functional templates. This modular design is a champion of:

  • Excellent Version Control: Track every change with precision.
  • Simplified Dependencies: Manage relationships between components with ease.
  • Stable, Reusable, and Maintainable Templates: Build once, reuse often, and update with confidence.

Imagine this: chatbot and code generation templates can share the same skeleton but seamlessly bootstrap different LLMs and model servers. For instance, you could configure one for Granite 3.1 with Llama CPP for chatbots, and another for Mistral and vLLM for code generation. The possibilities are vast!

Model Integration Strategies: Flexibility Meets Control 🎯

The session detailed three distinct and powerful strategies for integrating AI models:

  1. Starter Models: This is your rapid prototyping powerhouse. The software template bootstraps the model runtime and LLM directly. This offers minimal onboarding friction, providing developers with a functional model right out of the gate.

    • The Challenge: This approach often requires creating custom, optimized Docker images for deployments (e.g., on OpenShift). The fast-paced AI world means frequent updates, which can lead to dependency nightmares (imagine your model runtime not supporting your chosen model!). These images can also become quite bulky, consuming significant developer time and CI/CD capacity. While tools like Renovate can help automate updates, it’s an ongoing effort.

  2. Bring Your Own Model: This strategy empowers developers to bring their own model server endpoint and deployed model. It fosters innovation and experimentation, allowing teams to integrate any third-party model without being constrained by default bootstrapped options.

  3. Backstage Catalog: This is your go-to for production environments, offering a clean separation of models from templates. Platform engineers curate a list of models that meticulously adhere to organizational policies (security, compliance). A clever plugin acts as a bridge, connecting models from various model registries (like Hugging Face) to the Backstage Catalog. The software template then consumes this curated list, ensuring policy-compliant model integration.

A Demo That Delivers ✨

The live demo was a compelling showcase of these strategies in action:

  • Starter Models: Witnessed the creation of a chatbot application using Llama CPP and the Granite model. The process was remarkably smooth, involving selecting the model runtime, providing repository and deployment details, and then observing the automated CI/CD pipelines and GitOps deployments come to life. The application was successfully launched and tested, proving the “ready-to-go” promise.
  • Bring Your Own Model: The flexibility of this approach was evident as an external model server endpoint and model name were simply specified, demonstrating a seamless integration.
  • Backstage Catalog: The demo highlighted the OpenShift AI model catalog, where curated models like Granite are readily available. A Backstage plugin masterfully bridged this catalog to the Backstage catalog itself, which was then effortlessly selected within the software template for application creation. This beautifully illustrated the power of centrally managed, compliant models.

Navigating the Road Ahead: Challenges and Future Work 🛣️

While the progress is phenomenal, the journey isn’t without its bumps. The session candidly addressed some remaining challenges:

  • User Experience Refinements: The initial presentation of model strategies could be confusing, with feed labels sometimes leading to uncertainty. While effective workarounds were implemented, continuous refinement is a priority.
  • Environment Awareness Gaps: A lack of runtime awareness regarding installed platform technologies (like OpenShift AI) can lead to unexpected failures. For instance, selecting an option like creating an OpenShift AI workbench might not be available in the target environment.

The session concluded with a generous sharing of resources, providing links to repositories for the software templates, building blocks, and model catalog bridging tools. New blog posts detailing the software templates and AI asset catalog were also announced. The presenters expressed genuine enthusiasm for sharing this groundbreaking work and warmly invited feedback from the community.

This initiative by Red Hat, powered by Backstage Software Templates, is more than just a technical advancement; it’s a fundamental shift in how we approach AI development. By embracing standardization, abstraction, and modularity, teams can unlock unprecedented levels of productivity and focus on what truly matters: building the future of AI.

Appendix