As the title suggests, we are going to discuss the concept of load balancing a cluster of inexpensive load balancers to scale out SSL capabilities (or even Layer 7 operations...) rather than the typical idea of scaling up SSL offload capabilities by buying ever-larger appliances with ever more expensive hardware and license requirements.
What on earth am I suggesting?
The idea itself came out of a discussion (as so many good ideas do!) where we were discussing an SSL offload requirement that could potentially grow to the point that it needs 100,000 TPS or more, ouch!
It was brought up in a discussion that, to achieve that sort of TPS quotes had been $500,000+ for a highly available pair of capable appliances including the associated licenses to unlock the hardware...even bigger ouch!!
I couldn't help but think - couldn't you just use a large cluster of cheaper load balancers all being load balanced by a pair of fast Layer 4 load balancers keeping it transparent...? However, the problem would be managing such a beast...
But what if we used an API or central control node to manage it all...?
So the brainchild was born for better or worse...
What exactly am I proposing...?
As a picture speaks louder than words - I'm suggesting along the lines of this:
So, what are we doing exactly?
The plan, as I mentioned briefly already, is to use an inexpensive pair of load balancers using Layer 4 DR mode for maximum throughput and transparency to load balance some more mid to high-end load balancers handling the SSL offload at a much more affordable price than simply buying a big box.
The idea hinges on your ability to manage this effectively, because without an API or some other way to perform maintenance tasks across the whole farm of load balancers it would become cumbersome fast. However, with an API and some scripting knowledge, this should be fairly easy to overcome. And guess what...? We have an API already:
https://www.loadbalancer.org/blog/how-do-i-automate-load-balancer-deployments/
Time to get our DevOps on!
Here is an example simple bash script to leverage the API to perform actions on multiple load balancers at once.
#!/bin/bash
# Were we called with at least one arg?
if [ $# -eq 0 ]; then
echo "Not enough paramters: Usage - '$0 command' "
exit 42
fi
# Space separated list of IP addresses to push the command to
SERVERS="192.168.10.11 192.168.10.12 192.168.10.13 192.168.10.14"
# Pull in all args passed to this script as the required command
COMMAND=$*
for SERVER in ${SERVERS} ; do \
echo ssh root@${SERVER} ${COMMAND}
done
exit 0
Does it work on price? I think it does!
2x Enterprise 10G with 1 yr 24/7 Support = $16,980.00 (USD) Excl. Tax
9x Enterprise Ultra with 1 yr 24/7 Support = $152,910.00 (USD) Excl. Tax
Total = $169,890 (USD) Excl. Tax
Which could potentially handle ~ 150,000 TPS!
Check it out yourself using our online quote form: https://www.loadbalancer.org/quote
When else might this work?
I think the other thing to mention is that scaling SSL isn't the only thing that can be a problem, scaling L7 transformations can get tough, scaling WAF's can get even tougher and approaching either problem with this method would be just as applicable.
Why isn't everyone doing this?
Well, it does involve more work than an "off the shelf" solution but I think you will find that when you look beyond the typical enterprise customer at some of the largest deployments around that this techique is actually used fairly often...take Facebook for example:
https://www.bizety.com/2017/01/17/facebook-billion-user-load-balancing/
Final thoughts and should you?
Large load balancers with massive SSL offload capability are fine, they may suit some enterprises because they are simple to deploy and maintain...
However, when you have a service with requirements that are growing or you just chucked out those 2048 bit certs and deployed 4096 only to realise that your big pair of boxes are no longer up to the task you may value the extra investment of time at the beginning to have a system that only requires adding more Layer 7 load balancers as you grow.
I end with "should you?" and to answer this I think it's fair to say "Yes" - this is a great way to tackle the problem of scale.
While it does involve having the internal technical resources capable of implementing and maintaining such a solution the benefits if you are a company with a technical abundance will be great. You'll be saving a truck load of money while justifying your tech team's salaries... All this with the added technical benefit of having an endlessly scalable load balancer designed from the outset to grow with your needs.