eBPF-Powered Load Balancing
Implementing Hot Standby Load Balancing with eBPF and Golang
One of the features that recently caught my attention was custom load balancing for services utilizing SO_REUSEPORT. This not only allows multiple service instances share a single host port but also define custom load-balancing logic. In this post, we’re going to talk about this in depth and reference a simple code example using Golang for you to try it yourself.
⚠️ Note: There are many useful resources that explain the socket and eBPF basics better than I can, so I’m only going to focus on preparing the groundwork for the code example.
First of, what is SO_REUSEPORT and how does it work.
SO_REUSEPORT
Briefly, there are three main approaches how to configure you service to listen on the host port:
Single listen socket, single worker process.
Single listen socket, multiple worker processes.
Multiple worker processes, each with separate listen socket.
The last configuration is what you end up with if you configure your services to use SO_REUSEPORT. For more details, take a look at this post.
While this might seem really cool, by default you are constrained to a round-robin scheduling, meaning any time a packet is received on the port that has SO_REUSEPORT configured, the packets are evenly distributed among listening services. However, considering the nature of different load balancers, it would be great if we could define our own logic for how and when packets are transferred to each of the listening services.
This is exactly what we’re going to talk about.
Hot Standby Load Balancing
Although the load balancing algorithms range from simpler to complex, we’re gonna look at probably the most basic one, just for the sake of proof-of-concept illustration.
We’re gonna deploy an eBPF program to balance the traffic between two services — namely primary and standby. The idea is that you have two services running in parallel, but the standby service only receives traffic in case the primary service is down.
But how can we override the default round-robin scheduling? Let’s have a look.
Code Example
The program is two-fold:
Run two HTTP servers (namely
primaryandstandby), binded to the same port using SO_REUSEPORTLoad and attach an eBPF program to
sk_reuseporthook point which overrides the default round-robin load balancing of SO_REUSEPORT
⚠️ Note: There are many tools to deploy eBPF program, but I prefer
ebpf-goframework, so don’t feel constrained about it.
I find code example renders in Substack tedious, so I’ll refer to my GitHub repository with the code and test results.
Here’s the link.
As with my previous posts, I intentionally left the technical details in the code comments, which I believe makes the concepts slightly clearer.
I hope you find this resource as enlightening as I did. Stay tuned for more exciting developments and updates in the world of eBPF in next week's newsletter.
Until then, keep 🐝-ing!
Warm regards, Teodor