Benefits of load balancing HPE ClusterStor
Load balancing an HPE Cray ClusterStor deployment primarily involves balancing I/O and capacity across Object Storage Targets (OSTs), network paths (Multi-Rail LNet), and metadata/tiering resources, offering the following benefits:
- Elimination of I/O hot-spotting and tail latency: Distributing file stripes evenly across Object Storage Targets prevents individual drives or controllers from becoming I/O bottlenecks, eliminating the tail latencies and straggler issues that leave hundreds of expensive compute or GPU nodes idling at MPI synchronization barriers.
- Maximized aggregate throughput and hardware ROI: Balancing concurrent I/O streams fully saturates the PCIe and drive channels across multiple SSU storage controllers while distributing traffic evenly across Multi-Rail LNet fabric links (Slingshot or InfiniBand), preventing packet congestion and maximizing aggregate throughput.
- Balanced capacity utilization and flash lifespan: Capacity-aware load balancing prevents premature file-creation errors by ensuring individual OSTs do not fill up while others sit idle, while also promoting even wear leveling across NVMe targets and facilitating smooth automated tiering to bulk disk pools via ClusterStor Data Services.
About HPE ClusterStor
HPE ClusterStor is a high-performance parallel storage system purpose-engineered for HPC and AI workloads. It
is built around the Lustre open-source parallel file system and delivers extreme scalability, high throughput, and
low latency for large-scale compute clusters.
The solution can be configured with all-flash, hybrid, or disk-based nodes and supports multi-protocol access,
including SMB (CIFS) via dedicated Gateway (GW) nodes.
Why Loadbalancer.org for HPE ClusterStor?
Loadbalancer’s intuitive Enterprise Application Delivery Controller (ADC) is designed to save time and money with a clever, not complex, WebUI.
Easily configure, deploy, manage, and maintain our Enterprise load balancer, reducing complexity and the risk of human error. For a difference you can see in just minutes.
And with WAF and GSLB included straight out-of-the-box, there’s no hidden costs, so the prices you see on our website are fully transparent.
More on what’s possible with Loadbalancer.org.
How to load balance HPE ClusterStor
The HPE ClusterStor SMB Gateway (GW) nodes provide SMB access to the shared Lustre file system. The function of the load balancer is to ensure that client SMB connections are distributed across the healthy GW nodes, and to provide a single DNS name that clients use to access the SMB shares. This provides a highly available and scalable service: if a GW node fails, clients are directed to the remaining healthy nodes.
To provide HA for the load balancer, Loadbalancer.org recommends that 2 appliances are deployed as an HA clustered pair.
For full details, refer to the HPE ClusterStor deployment guide below.
HA & virtual service (VIP) requirements
Load balancing HPE ClusterStor requires no Virtual Services (VIPs). Instead, both load balancers are configured
as smart DNS name servers for the FQDN that clients use to access the SMB shares (e.g. clusterstor.example.com). This is achieved using the load balancer’s built-in GSLB service and by using DNS delegation.
The GSLB service uses weighted round-robin load balancing to distribute inbound client connections across the
healthy GW nodes. The weights for the nodes can be set as required depending on relative node performance.
Because clients connect directly to the GW nodes, the load balancer is not in the path of the SMB traffic. This
means that throughput is limited only by the network between the clients and the GW nodes, which makes this
method well suited to high-throughput HPC and AI storage workloads.
Load balancing deployment concept
The diagram below illustrates how the load balancer is deployed:
- The client sends a DNS query for the service FQDN, e.g. clusterstor.example.com to the local DNS server.
- The local DNS server has the sub domain delegated to both load balancers (both load balancers are
configured as name servers for the sub domain). - One of the load balancers receives the delegated DNS query.
- The load balancer selects a healthy GW node based on the GSLB health checks and the round robin
algorithm used. - The load balancer returns the IP address of the selected GW node to the DNS server.
- The DNS server returns the IP address of the selected GW node to the client.
- The client connects directly to the SMB share on the GW node.
