Presenters

Source

CruiseCube: Revolutionizing Kubernetes Resource Optimization 🚀

Are your Kubernetes clusters costing you more than they should, even when you try to be frugal with resource requests? You’re not alone! Many organizations struggle with the fear of downtime caused by under-provisioning, leading to a cycle of over-provisioning that inflates costs and leaves nodes underutilized. But what if there was a smarter way? Enter CruiseCube, an innovative open-source tool designed to tackle pod-level resource optimization head-on.

The Kubernetes Cost Conundrum: Why Optimization is Tricky 💰

The core problem, as highlighted by Shubham Rai and Raman Tehlan, stems from a fundamental fear: CPU throttling and kills. If you underestimate memory needs, your pod gets terminated, leading to business downtime. This fear drives developers to over-provision both CPU and memory. Adding to this, the manual process of analyzing and updating YAML files for optimization is tedious and often overlooked.

This over-provisioning at the request level cascades to the node level, resulting in underutilized nodes. This, in turn, forces cluster autoscalers into making inefficient and costly decisions. While node-level autoscaling seems like a solution, it can’t fix the root problem of inefficient pod-level requests.

CruiseCube vs. VPA: A New Approach to Optimization 🎯

Kubernetes already offers tools like the Vertical Pod Autoscaler (VPA). However, CruiseCube distinguishes itself through its optimization model and cluster view.

  • Optimization Model: VPA relies on periodic analysis and recommendations that aim to be valid for a significant duration. CruiseCube, on the other hand, focuses on continuous, feedback-driven optimization with a very limited time horizon. It leverages real-time pod usage to make recommendations.
  • Cluster View: VPA makes decisions at the workload level, potentially across distributed pods on different nodes. CruiseCube operates at the individual pod level within its own node context. This node-local view is crucial for understanding resource dynamics.

The Tech Behind the Magic: What’s Changed? ✨

Two key advancements in Kubernetes have paved the way for CruiseCube’s capabilities:

  1. In-Place Pod Resource Update (GA in Kubernetes 1.27): This feature allows updating resources for a pod without a restart. This minimizes disruption and enables fine-grained, pod-level decisions directly on the node where the pod resides.
  2. Pressure Stall Information (PSI) Metrics (Beta in Kubernetes 1.26): PSI provides contention information per container on each node. This allows for right-sizing even without explicit CPU limits, offering valuable insights into actual resource pressure.

CruiseCube leverages these signals – CPU usage and CPU PSI metrics – to determine the optimal CPU requests for pods.

The Importance of Node Context: Sharing the Headroom 🤝

Raman explains that decisions are often taken at the node level because multiple pods share a single node, each with varying headroom requirements. Headroom is the difference between a pod’s burst capacity and its base requirement. Since CPU workloads tend to burst infrequently but significantly (up to 10,000 times), CruiseCube can intelligently share this headroom space among pods on the same node. By taking the maximum headroom requirement across all pods on a node, CruiseCube reserves that space, allowing individual pods to burst as needed.

How CruiseCube Works: A Continuous Optimization Loop 🔄

CruiseCube operates on a continuous loop strategy with several stages:

  1. Observation: Collecting stats and historical metrics from the cluster, which are then stored in a database.
  2. Learning: Analyzing historical data to identify patterns, like recurring peaks, to inform recommendations.
  3. Recommendation & Application: Generating recommendations for workloads and safely applying these changes to the cluster. This is a short-term, frequent optimization process.

CruiseCube applies optimizations in two key phases:

  • Time of Admission: An admission webhook intercepts new pod provisioning. It consults historical data for that workload or pod and applies recommendations immediately. If a previous pod was killed due to throttling, this is learned, and recommendations are applied proactively.
  • Continuous Optimizer: This component runs periodically, scanning all workloads and applying in-place, non-disruptive recommendations.

CruiseCube’s Architecture: The Engine Room 🛠️

  • Prometheus: The source of all metrics and stats.
  • Controller: Orchestrates the entire process, collecting and transforming stats.
  • Statistics Engine: Continuously generates stats for workloads as new data arrives.
  • Runtime Optimizer: Reads stats, generates recommendations, and applies them.
  • Mutation Webhook: Intercepts new pod provisions and applies recommendations based on historical data.
  • Dashboard: Provides a user-friendly front-end for monitoring and control.

Built-in Guardrails: Safety First! 🛡️

CruiseCube understands the criticality of your workloads. It includes robust guardrails:

  • Recommend Only Mode: By default, recommendations are not applied. You must explicitly enable “Cruise Mode” for a workload.
  • Criticality-Aware Eviction: CruiseCube prioritizes less critical workloads for potential adjustments, while highly critical workloads are protected. You can also customize priority levels.
  • Feedback Loop for OOM Kills: CruiseCube learns from past Out-Of-Memory (OOM) kills, analyzing node pressure and memory patterns to prevent recurrence.

The Dashboard: Your Command Center 📊

The CruiseCube dashboard offers a comprehensive view:

  • Cost Savings Tracking: Monitor current and potential cost savings.
  • Resource Overview: Visualize allocatable, requested, and recommended CPU and memory across your cluster.
  • Historical Timelines: Track resource changes over time.
  • Workload Management: View recommendations for individual workloads, enable/disable Cruise Mode, and track applied changes.
  • Policy Management: Configure optimization policies and disruption windows.

Best Use Cases and Considerations 🤔

CruiseCube is ideal for:

  • Stateless Workloads: Especially in dev and staging environments.
  • Short-Lived Jobs and Services: Where minimal disruption is key.
  • Recommendation Mode: For stateful applications (databases, caches, queues) and latency-sensitive workloads, using CruiseCube in recommendation mode and manually tuning is a safe approach.

CruiseCube is continuously evolving, with ongoing work to support workloads with Horizontal Pod Autoscaler (HPA) enabled.

Join the Movement! 🧑‍💻

CruiseCube is an open-source effort, and the team encourages you to star the repository, explore the code, and contribute. Install it on your cluster in recommendation mode to see firsthand how much you can save! Reach out on Discord or connect with Shubham and Raman for any questions.

By intelligently optimizing pod-level resource requests, CruiseCube empowers you to reduce costs, improve cluster efficiency, and maintain application stability. It’s time to move beyond the fear of under-provisioning and embrace a smarter, more cost-effective approach to Kubernetes resource management!

Appendix