Load balancing your load balancers for endless scalability

As the title suggests, I'm going to take a look at the concept of load balancing a cluster of inexpensive load balancers to scale out SSL capabilities (or even Layer 7 operations), rather than the typical idea of scaling up SSL offload capabilities by buying ever-larger appliances with ever more expensive hardware and license requirements.

Light bulb moment

The idea itself came out of a conversation, as so many good ideas do. We were discussing an SSL offload requirement that could potentially grow to the point of needing 100,000 TPS or more - ouch!

Somebody shared that to achieve that sort of TPS, they'd been quoted $500,000+ for a highly available pair of capable appliances, including the associated licenses to unlock the hardware - even bigger ouch!!

I wondered: couldn't you just use a large cluster of cheaper load balancers, all being load balanced by a pair of fast Layer 4 load balancers, keeping it transparent? Managing such a beast wouldn't be easy. But what if we used an API or central control node?

So the brainchild was born, for better or worse. As a picture speaks louder than words, I'm suggesting along the lines of this:

Creating a monster

The plan is to use an inexpensive pair of load balancers - using Layer 4 DR mode for maximum throughput and transparency - to load balance some more mid to high-end load balancers. This enables us to handle the SSL offload at a much more affordable price than simply buying a big box.

The idea hinges on your ability to manage this effectively, because it could quickly become cumbersome without an API or some other way to perform maintenance tasks across the whole farm of load balancers. With an API and some scripting knowledge, however, this should be fairly easy to overcome. And guess what? We have an API already:

/blog/how-do-i-automate-load-balancer-deployments/

Time to get our DevOps on!

Here is an example simple bash script to leverage the API to perform actions on multiple load balancers at once.

#!/bin/bash

# Were we called with at least one arg?
if [ $# -eq 0 ]; then
   echo "Not enough paramters: Usage - '$0 command' "
   exit 42
fi

# Space separated list of IP addresses to push the command to
SERVERS="192.168.10.11 192.168.10.12 192.168.10.13 192.168.10.14"

# Pull in all args passed to this script as the required command
COMMAND=$*

for SERVER in ${SERVERS} ; do \
   echo ssh root@${SERVER} ${COMMAND}
done

exit 0

Does it work on price? I think it does!

2x Enterprise 10G with 1 yr 24/7 Support = $16,980.00 (USD) Excl. Tax
9x Enterprise Ultra with 1 yr 24/7 Support = $152,910.00 (USD) Excl. Tax

Total = $169,890 (USD) Excl. Tax

Which could potentially handle ~ 150,000 TPS!

Check it out yourself using our online quote form

When else might this work?

I think the other thing to mention is that scaling SSL isn't the only thing that can be a problem. Scaling L7 transformations can get tough, scaling WAFs can get even tougher and approaching either problem with this method would be just as applicable.

Why isn't everyone doing this?

Well, it does involve more work than an "off the shelf" solution. But I think you will find that when you look beyond the typical enterprise customer at some of the largest deployments around, this techique is actually used fairly often. Take Facebook for example:

https://www.bizety.com/2017/01/17/facebook-billion-user-load-balancing/

To try or not to try

Large load balancers with massive SSL offload capability are fine. They may suit some enterprises because they are simple to deploy and maintain.

However, when you have a service with requirements that are growing - or you just chucked out those 2048 bit certs and deployed 4096 only to realise that your big pair of boxes are no longer up to the task - it could be worth doing something different. You may value the extra investment of time at the beginning to have a system that only requires adding more Layer 7 load balancers as you grow.

So should you try it? I think it's fair to say "yes" - this is a great way to tackle the problem of scale.

While it does involve having the internal technical resources capable of implementing and maintaining such a solution, the benefits if you are a company with a technical abundance will be great. You'll be saving a truckload of money while justifying your tech team's salaries. And you get the added technical benefit of having an endlessly scalable load balancer designed from the outset to grow with your needs.