Presenters

Source

Grafana: Beyond Observability - Powering Business Intelligence and Analytics 🚀

Hey tech enthusiasts! Ever wondered if that powerful tool you use for keeping an eye on your systems could also be the key to unlocking deeper business insights? At Grafana Labs, we’ve been doing just that, transforming Grafana from a go-to for SREs and engineers into a central hub for our business data. Join us as we dive into how we’ve built Grafana dashboards for Business Intelligence (BI) internally, creating a versatile visualization layer on top of our entire business data stack.

Meet the Speakers 👨‍💻

  • Sam Jewell: A software engineer at Grafana Labs, Sam has been instrumental in building semantic layer support into Grafana.
  • Chris: One of the earliest data hires at Grafana Labs, Chris leads the central data and analytics team, bringing over two years of experience in this domain.

Our “Why”: Expanding Grafana’s Reach 💡

You’re likely already using Grafana for its incredible monitoring capabilities, especially when things go wrong. It’s a flexible tool, but have you considered its potential beyond technical operations? At Grafana Labs, every department leverages Grafana, from product management to sales and marketing. We use it for:

  • Product Analytics: Understanding feature adoption – “Is my feature being used?” 🤔
  • Project Management: Identifying bottlenecks for engineers – “Are our engineers getting blocked?” 🚧
  • Business Intelligence (BI): Guiding sales efforts – “Which prospects should I be calling today?” 📞
  • Observability: Our core function – “Is my service up?” 🚨

So, why Grafana for everything?

Unique Motivations (and Universal Benefits) ✨

While our position as Grafana creators gives us a unique advantage (we’re essentially dogfooding our own product!), many of our reasons resonate with any organization:

  • Reduced Complexity & Cost: Consolidating tools simplifies operations and saves money. 💰
  • Seamless Collaboration: Empowering almost all staff to operate within the same tool fosters cross-departmental collaboration and visibility into each other’s work. 🤝
  • Grafana’s Strengths:
    • Big Tent of Data Sources: Query and visualize data from multiple sources side-by-side, even blending them in a single panel. 🌐
    • Alerting: Set alerts on anything. 🚩
    • Performance: Blazing fast and low-latency, handling massive data streams with ease. ⚡

We’re excited to share some real-world examples of our dashboards and pipelines, hoping to inspire you to explore Grafana’s potential in your own business context!

Demo Dive: Visualizing Product Adoption Funnels 📊

Let’s explore some concrete examples! All dashboards are available on play.grafana.org and learn.grafana.net.

Kubernetes Monitoring Adoption Funnel 📈

This dashboard was built for our product teams (design and engineering) to visualize and understand the Kubernetes monitoring adoption funnel. It tracks four key stages: awareness, activation, engagement, and our “aha” moment.

  • End-User Empowerment: Product teams can inspect funnel performance daily, identifying drop-off rates, pull-through rates, and key KPIs like adoption percentage, time to setup, and time to value.
  • Automated Annotations: We overlay annotations directly onto the dashboards, triggered by events like A/B test launches. This is powered by BigQuery data, eliminating manual annotation efforts. For instance, we can see how launching an A/B test to 30% of our population impacted a metric, and then observe the return to baseline when rolled out to 100%.
  • Interactivity: Dashboards leverage data links and dashboard variables to allow users to slice and dice data by monthly cohorts, drilling down into underlying data with ease.

A/B Testing Measurement & Grafana Assistant 🤖

This dashboard is crucial for our growth and onboarding teams, tracking A/B tests over the last two to two-and-a-half years. It combines BigQuery data with metadata about test objectives and outcomes.

  • Context for AI: This rich historical data is invaluable for our AI agents, like the Grafana Assistant. We can now ask the Assistant to perform A/B testing measurements directly.
  • LLM Power-Up: Instead of manual analysis or specialized tools, we pass the dashboard context to the Assistant. It can parse the underlying panels and JSON, identify the correct statistical tests (like a two-proportion Z-test), and even execute these calculations within BigQuery.
  • Slack Integration: The real inflection point came when we integrated the Grafana Assistant into our Slack channels. This allows our teams, who heavily rely on Slack, to access data, dashboards, and analyze conversations, seamlessly embedding the Assistant into our daily workflows.
  • Assistant Configuration: To enable this, we configure the Assistant’s SQL table discovery, syncing metadata from our data sources (like BigQuery) and our semantic layer into a vector database. This allows the Assistant to perform more efficient queries over metadata.

Grafana as Our BI Tool: Empowering Sales 🎯

Let’s shift gears to sales analytics, showcasing how Grafana serves as our Business Intelligence tool.

The Sales Pipeline Dashboard 💰

This dashboard, available on play.grafana.org with synthetic data (but identical internal schema and panels), is designed for our go-to-market teams (salespeople and SDRs).

  • Data Scale: Our internal version loads hundreds of thousands of records from Salesforce, including prospects and contacts, with over a hundred columns per record.
  • High-Level Overview: The dashboard provides a top-level view of the entire sales pipeline, broken down by account size, sales funnel stage, activity levels, and progress towards quarterly targets.
  • Slice and Dice Capabilities: With over 15 dimensions at the top, executives can focus on their regions, while frontline sellers can filter down to the few accounts relevant to their daily or weekly pursuits.

Introducing the Semantic Layer: A Game Changer 💡

While building the sales dashboard, we identified opportunities to further enhance the user experience. This led to the implementation of a semantic layer.

What is it? It sits in front of your data warehouse, connects to Grafana, and offers three significant improvements:

  1. Interactive Filtering & Drill-Down: Click data within panels to filter and drill through dashboards seamlessly.
  2. No SQL Required: Once set up, you don’t need to write SQL or understand complex data models.
  3. Elevated AI Capabilities: Enhances what AI agents like the Grafana Assistant can do with your data.

The Impact of the Semantic Layer:

  • Before (SQL Data Source): Attempting to filter directly within SQL queries proved brittle, often breaking complex queries. This made interactive filtering impossible. Building panels required writing ~30 lines of SQL, understanding table joins, and manually referencing filters – a time-consuming and maintenance-heavy process.
  • After (Semantic Layer):
    • Intuitive Filtering: Click on data points, and the entire dashboard filters instantly. Drill down through engagement signals and activity pulses to pinpoint specific accounts to focus on. This dramatically speeds up data exploration and insight generation.
    • Simplified Panel Creation: Building a panel now involves just a few selections, with the semantic layer generating the necessary SQL dynamically. This not only speeds up development but also vastly lowers the barrier to entry, allowing anyone in the business to query data without learning SQL or the data model.
    • Dynamic SQL Generation: The semantic layer intelligently injects necessary joins and traverses the table graph to apply filters dynamically. This overcomes the limitations of static SQL queries.

Why Not Just Rely on LLMs for SQL? 🤔

While generative AI is powerful, LLMs can “hallucinate” and may not grasp the nuances of your specific data model or business definitions.

  • Source of Truth: The semantic layer is configured via YAML files, storing metadata like descriptions, titles, and aggregation logic. Crucially, it acts as a single source of truth for business metrics (e.g., how your company defines “Total ARR”). This ensures consistency across all dashboards and AI interactions.
  • Agent Enhancement: When asking the Grafana Assistant a question like “how many customers are there in Amir?”, using the semantic layer (powered by Cube) results in fewer, faster, and more cost-effective queries compared to directly querying BigQuery. It leverages that single source of truth for accurate and granular answers.

The Technology Behind It: Cube.js 📦

We’ve built a data source that wraps Cube.js, an open-source project. You can download and self-host Cube.js today, and the data source itself is also free and open-source. You can find the repositories on GitHub and we encourage you to try it out in your self-hosted or cloud Grafana instances!

Join Us & Q&A! 🗣️

Chris and I will be at the “Ask the Experts” session tomorrow at 3:15 PM. The demos shown today are available at play.grafana.com. We’re excited to answer any questions you might have!

(Audience applauding)

Appendix