Presenters

Source

From Vanilla gRPC to DRPC: Scaling Microservices at DigitalOcean 🚀

Ever wondered how cloud giants handle the immense complexity of running thousands of services, all while keeping things lightning-fast and reliable? Today, we’re diving deep into the world of gRPC and how DigitalOcean has not only adopted it but evolved it into their own robust solution, DRPC. Get ready for a journey from the basics of remote procedure calls to the nitty-gritty of their custom platform!

The Magic of Remote Procedure Calls (RPC) ✨

At its core, RPC is all about making a function call on a remote machine feel as simple as calling a local one. Think of it like having a universal phone book (your DNS or, in DigitalOcean’s case, their Endpoint Discovery Service - EDS) that lets your services connect and communicate seamlessly, no matter where they reside.

Why gRPC? Enter HTTP/2 and Protocol Buffers! 🌐

If RPC is so great, why did Google invent gRPC? Two words: speed and efficiency.

💨 The Need for Speed: HTTP/2 Revolution

Remember the old days of HTTP/1? Each request meant a new, clunky TCP connection, and you had to wait for one response before sending the next. Slow, right? Even HTTP/1.1’s pipelining had its limitations.

Then came HTTP/2, a game-changer! It introduces:

  • Single TCP Connection: Multiple requests and responses flow simultaneously over one connection.
  • Streams: These independent streams don’t wait for each other, making communication much faster.
  • Server Push: Servers can proactively send updates to clients without being asked.

💡 Efficiency Matters: Protocol Buffers (Protobuf)

Before Protobuf, inter-service communication was a chore:

  • Manual Serialization/Deserialization: Developers had to write code to convert data to and from formats like JSON or XML. This was tedious, error-prone, and bulky.
  • Bulky Data: JSON and XML are verbose, leading to larger message sizes and slower transfers.
  • Language Barriers: Making services written in different languages (like Java and C++) talk to each other was a significant challenge.

Protobuf swoops in to save the day by:

  • Strictly Typed Definitions (.proto files): A single source of truth ensures all services speak the same language, making communication language-agnostic.
  • Automatic Serialization/Deserialization: Developers are abstracted from the low-level details, simplifying the coding process.
  • Efficient Binary Encoding: Messages are highly compressed, similar to zipping files, making them smaller, faster to transmit, and more efficient.

The gRPC Workflow: Code Generation and Stubs 🛠️

gRPC leverages an Interface Definition Language (IDL), typically .proto files. These files define your services and message structures. Command-line tools then generate client and server code (stubs) in your preferred language.

Here’s how it typically flows:

  1. Your Application: Interacts with a client-side stub.
  2. gRPC Client Stub: Translates your application’s calls into a format gRPC understands.
  3. gRPC Library: Sits on top of HTTP/2, handling the network communication.
  4. Transport Layer: The underlying network infrastructure.

The same process happens in reverse on the server-side, ensuring seamless communication.

DigitalOcean’s Scale: Numbers That Amaze 📊

DigitalOcean operates at an incredible scale:

  • 4,000+ Unique Services: Constantly running.
  • 25,000+ Service Instances: Managing this massive deployment.
  • Kubernetes-Powered: Their platform is built on Kubernetes.
  • 1 Million Requests Per Second: Internal traffic volume.
  • Sub-100ms Latency: For customer-facing services.
  • Sub-10ms Latency: For internal communications.
  • 15-20 GB Data Per Second: Moving data across their network.
  • 99.9% gRPC Adoption: Almost all internal services rely on gRPC (or DRPC). Only client-facing services might use JSON for compatibility.

From Vanilla gRPC to DRPC: The DigitalOcean Way 💡

While vanilla gRPC is powerful, DigitalOcean needed more. They developed DRPC (DigitalOcean RPC), a framework built on top of gRPC, to address their specific needs for reliability and scalability.

Key Features of DRPC: Out-of-the-Box Goodies 🎉

DRPC provides a wealth of features automatically, saving developers immense time and effort:

  • Structured Logging: Consistent and detailed logs for better debugging.
  • Tracing: End-to-end request tracing for performance analysis.
  • Metrics: P50, P99 latency, request counts, error rates, and more, readily available.
  • Client Load Balancing: Built-in support for various load balancing strategies (round robin, weighted round robin, custom).
  • TLS Everywhere: Automatic TLS encryption for all communications.
  • mTLS Configuration: Simplified mutual TLS setup for secure service-to-service authentication.
  • RBAC (Role-Based Access Control): Fine-grained control over who can access which RPCs.

The Power of Interceptors 🎛️

gRPC is highly extensible, and DRPC leverages this through interceptors. These are middleware components that can hook into the request/response lifecycle. DRPC uses interceptors extensively for:

  • Context Propagation: Adding crucial information like request IDs, user agents, and user IDs into the gRPC context, making it available throughout the call chain.
  • Logging & Tracing: Integrating with OpenTelemetry for robust tracing and observability.
  • Metrics Collection: Automatically gathering performance metrics.
  • Security: Implementing mTLS and RBAC checks.

Monorepo: The Unsung Hero? 💾

DigitalOcean heavily relies on a monorepo for managing their .proto files and generated code. This approach allows services to easily import and share definitions, simplifying inter-service communication. While initially met with skepticism, it has proven to be a significant advantage for their microservice architecture.

DOCCC: Their Kubernetes Platform ☁️

DRPC is seamlessly integrated with DOCCC, DigitalOcean’s custom Kubernetes platform. With just a few configurations like auto_tls: true and specifying service accounts, DOCCC handles the heavy lifting of TLS encryption and certificate management for DRPC services.

EDS: The Intelligent Service Discovery 📡

Endpoint Discovery Service (EDS) acts as the “DNS” for DigitalOcean’s microservices. When a service needs to communicate with another, it queries EDS with the service name and desired region to get the available endpoints. DRPC integrates with EDS, making it feel like a natural DNS lookup for service discovery.

The Future of DRPC at DigitalOcean 🚀

By building DRPC on top of gRPC and integrating it with their custom platforms like DOCCC and EDS, DigitalOcean has created a highly scalable, reliable, and efficient microservice ecosystem. They’ve seen no issues so far, indicating that this approach is well-positioned for continued growth and innovation.

The journey from understanding RPC basics to implementing a sophisticated, in-house solution like DRPC is a testament to the power of thoughtful engineering and the adaptability of modern technologies. It’s a story that inspires us to look beyond standard solutions and build what truly works for our own unique challenges!

Appendix