Presenters

Source

Streamlining Kubernetes Cluster Management with Backstage 🚀

Managing Kubernetes clusters can quickly become a complex and time-consuming endeavor, especially as the number of clusters grows. In this insightful session, Alex Speranza, a DevOps and Platform Engineer at Informatica, shares how they leveraged Backstage to automate and simplify the process of onboarding Kubernetes clusters, transforming a convoluted workflow into a streamlined, user-friendly experience.

The Challenge: Scaling Cluster Management 📈

The core problem Alex and his team faced was the unforeseen rapid growth in their managed Kubernetes clusters. What was initially estimated to be a once-a-year task for the infrastructure team quickly escalated. In just two years, their cluster count surged from a mere three to a staggering 22. This exponential growth highlighted the need for a more efficient and automated approach to cluster onboarding and management.

The existing process involved a significant amount of back-and-forth between various systems: the Kubernetes cluster itself, the binder repositories (both infrastructure and application), and GitLab CI/CD. This manual, multi-step process was not only time-consuming but also prone to errors and difficult to scale.

The Solution: Backstage as the Central Hub ✨

Backstage emerged as the ideal platform to consolidate and automate these complex workflows. The goal was to create a single pane of glass where teams could manage their Kubernetes infrastructure with ease.

1. Integrating Rancher with Backstage 🌐

A significant hurdle was the lack of a direct plugin to integrate Rancher with Backstage. To overcome this, Alex’s team utilized the Rancher API to create a custom entity provider. This provider allows Backstage to discover and register clusters managed by Rancher as catalog resources. A key feature added was a direct link to the Rancher dashboard from the Backstage catalog, providing immediate access to cluster details.

2. Automating Infrastructure Binder Deployment 🛠️

Once a cluster is registered in Backstage, the next crucial step is configuring it with the infrastructure binder. This binder is a collection of essential applications like Prometheus (for monitoring), Grafana, and the Seed Secrets Operator.

The challenge here was to install and deploy the infrastructure binder without storing sensitive tokens directly. The team devised a creative solution:

• GitLab Agent Creation: A custom action within Backstage generates a key from the GitLab repository.
• Cluster-Side Job: A job executed by the cluster where Backstage is deployed then uses this key to run an helm install command. This command creates the GitLab agent, establishing a secure connection between the cluster and the infrastructure binder repository.

This method ensures that the infrastructure binder is deployed securely and automatically, without the need for manual token management.

3. Seamless Application Binder Onboarding 🚀

Following the infrastructure binder, the application binder is deployed, which contains the specific workloads for development teams. The process is similar:

• Token Generation: A token is generated from the application binder repository using the same action as for the infrastructure binder.
• Values File Integration: This token is then inserted into a values.yaml file within the infrastructure binder.
• Automated CI/CD: This allows GitLab CI/CD to automatically deploy the application binder onto the cluster.

This automation ensures that the application binder is automatically linked to the cluster, enabling its CI/CD pipeline to deploy autonomously without human intervention.

4. Enforcing Deployment Rules with Custom Pickers 🎯

A critical aspect of their deployment model is ensuring that:

• Only one infrastructure binder configures a cluster.
• A cluster cannot be used for application deployment until it’s configured with the infrastructure binder.

To enforce these rules, Backstage’s catalog was enhanced with a custom picker. This picker allows users to select clusters based on their configuration status:

• Non-configured clusters only: For initial infrastructure setup.
• Configured clusters only: For application deployment by dev teams.
• All clusters: For actions that don’t require specific configuration status.

This custom picker filters the clusters effectively, ensuring the correct deployment sequence and preventing misconfigurations.

5. Simplifying Secret Sealing with Backstage 🤫

The Seed Secrets Operator introduced an additional overhead for development teams, requiring them to manually seal secrets. The traditional workflow involved installing kubectl and kubeseal CLIs, retrieving cluster certificates, creating secret manifests, sealing them, and then updating the binder repositories.

To eliminate this complexity, Alex’s team implemented a Backstage-native secret sealing solution:

• Microservice for Sealing: A dedicated microservice handles the secret sealing process.
• Backstage UI Integration: Users can now seal secrets directly from the Backstage UI.
• Annotation-Based Public Key: The process leverages a simple annotation on the binder resource, which contains the public key of the cluster. This eliminates the need for users to retrieve cluster certificates or install new tools.
• No Private Key Exposure: Crucially, private keys are never exposed, ensuring enhanced security.

This new process makes secret management significantly easier and more accessible for developers, removing a major friction point.

The Impact: A More Linear and Efficient Workflow 🌟

The implementation of Backstage has led to a more linear and significantly less convoluted process. The need for constant back-and-forth between version control systems, Rancher, and GitLab repositories has been drastically reduced.

The DevOps team now primarily interacts with Backstage to:

• Register the infrastructure binder.
• Create the application binder, with the help of annotations that streamline secret management.

This transformation has not only saved valuable time and resources but also empowered developers by providing them with a user-friendly platform for managing their Kubernetes deployments.

Future Automation 🤖

Looking ahead, Alex’s team plans to automate even more processes within Backstage. A recent addition includes a template to create and register new clusters directly on Rancher, further simplifying the initial setup for the infrastructure team.

This initiative by Alex Speranza and his team is a fantastic example of how Backstage can be a powerful tool for streamlining complex DevOps workflows, making technology more accessible and efficient for everyone involved.

Appendix