At Ubicloud, we recently announced our Load Balancer (LB) service. This service distributes traffic across a set of VMs and provides high availability through health checks. Also, our Basic Plan comes at at no charge. This way, you can start building web apps using our LB, VMs, and Postgres service at a fraction of the cost of other cloud providers.

We like Ubicloud Load Balancer's simplicity in design and implementation. For our networking stack, we use Linux nftables. By extending on that implementation, we could build a scalable and flexible implementation in less than 1,000 lines of code. The core load balancer logic is just 8 lines.

This blog post shares our requirements, provides background on Ubicloud’s networking architecture, and then describes our LB implementation.

Load Balancer Requirements

Elastic Load Balancers come in various flavors and address a diverse set of requirements on the cloud. For Ubicloud, we decided to start with the smallest set of requirements that our users would need to deploy web applications. This meant providing:

• Common load balancing algorithms to balance requests across VMs
• Health checks to assess each VM’s state
• High availability through distributed load balancing and health checks
• Single endpoint for clients to connect
• Security through encrypting all traffic
• Cost efficiency

Ubicloud Networking Background

Ubicloud’s architecture has a control plane and data plane. The control plane holds the data model, responds to web requests, and coordinates changes to the data plane.

The control plane also “starts up” the data plane. Initially, the data plane consists of bare metal Linux machines. The control plane cloudifies these bare metal machines by installing required libraries. We refer to a cloudified bare metal machine as a VM host.

On the networking side, Ubicloud reprograms the Linux kernel in each VM host. This way, Ubicloud creates a network overlay on VM hosts. This includes the following.

• For each VM on a host, we set up a new network namespace.
• We add routing table entries to the host, so connections to the VM’s public IP address are routed to the network namespace.
• We set up Linux nftables in the namespace to perform NAT and packet filtering.
• Packets are then delivered to the VM through a TAP device.

Design & Implementation

This architecture lends itself nicely to providing a simple and scalable load balancing service.

On the data plane side, when customers put two VMs behind a single load balancer, we already have two network namespaces on different hosts. These namespaces handle NAT and firewalls. We introduce a new NAT table to selectively reroute connections to one of the VMs.

Our load balancer then is a set of extra nftables rules. We don’t run an additional process or node. Instead, the VM hosts act as load balancing network switches. Therefore, if you have four VMs behind a load balancer, you effectively have four load balancing nodes.

These nodes are tied together with a single DNS name, provided to the customer. The client can connect to any load balancing node, which then forwards the flow to the appropriate VM based on the algorithm and connection state.

For example, the following Linux netfilter rules load balance each connection in a round-robin fashion.

chain prerouting {
type nat hook prerouting priority dstnat; policy accept;
	ip daddr VM1_PUBLIC_IP tcp dport LB_SRC_PORT ct state established,related,new dnat ip to numgen inc mod 2 map { 0 : VM1_PRIVATE_IP . LB_DST_PORT, 1 : VM2_PRIVATE_IP . LB_DST_PORT }
}
chain postrouting {
	type nat hook postrouting priority srcnat; policy accept;
	ip daddr VM2_PRIVATE_IP tcp dport LB_DST_PORT ct state established,related,new snat ip to VM1_PRIVATE_IP
}

The following diagram shows the whole connection flow on the data plane side.

This design provides distributed load balancing on the data plane side. However, we still need to detect VM and host unavailability and have our load balancer perform failovers.

For this, we rely on the control plane. Our control plane already monitors VM health, making it the logical place to integrate load balancer health checks. By default, the control plane sends a health probe every 30 seconds and waits for 15 seconds for the VM to reply. If there are 3 consecutive failures, the control plane considers the VM unhealthy for load balancing purposes and updates necessary state changes on the data plane side.

All in all, these changes took less than 1,000 lines of code, where more than half of that code included our unit tests. Ubicloud is an open source project; and you can see our load balancer implementation in GitHub.

Conclusion