Direct Routing aka. Direct Server Return — How to solve the ARP problem on Windows Server 2022

Direct Routing aka. Direct Server Return (DSR) aka. N-Path is a great load balancing method. And it is without doubt the fastest method possible.

So, how does Direct Routing work?

First, the incoming traffic from the client hits the Virtual IP (VIP) on the load balancer.

Then the load balancer simply changes the destination MAC address of the packet (to one of the destination real servers in the pool) and flips it back to the switch which duefully delivers the packets to the selected real server:

You can see the problem here can't you?

The packet will say "Hello are you the VIP?"
Then the real server will say, "Get lost, no I'm not!".

Which is clearly not going to work...

So, how do you solve the Direct Routing problem on Windows Server?

Basically, we just need to add a loopback adapter with the VIP address. This then enables the real server can accept the packet...

Hang on, won't that confuse the network?

Yes, it will drive your network mad if you start sending out ARP notifications from multiple machines on the same IP address. This is called the ARP problem:

"THE SERVER MUST NEVER TELL THE NETWORK ABOUT THIS FAKE IP"!

And how do you solve the ARP problem on Windows Server?

Windows Server 2012 and later support Direct Routing (DR) mode through the use of the Microsoft Loopback Adapter that must be installed and configured on each load balanced (Real) Server. The IP address configured on the Loopback Adapter must be the same as the Virtual Service (VIP) address. This enables the server to receive packets that have their destination set as the VIP address. If a Real Server is included in multiple DR mode VIPs,
an IP address for each VIP must be added to the Loopback Adapter.

In addition, the strong/weak host behavior must be configured on each Real Server. The weak host model allows packets with any IP to be sent or received via an interface. The strong host model only allows packets with an IP belonging to the interface to be sent or received.

IMPORTANT: The following 3 steps must be completed on all Real Servers associated with the VIP.

Step 1 of 3: Install the Microsoft Loopback Adapter

1. Click Start, then run hdwwiz to start the Hardware Installation Wizard.
2. Once the Wizard has started, click Next.
3. Select Install the hardware that I manually select from a list (Advanced), click Next.
4. Select Network adapters, click Next.

5. Select Microsoft & Microsoft KM-Test Loopback Adapter, click Next.
6. Click Next to start the installation, when complete click Finish.

Step 2 of 3: Configure the Loopback Adapter

1. Open Control Panel and click Network and Sharing Center.
2. Click Change adapter settings.
3. Right-click the new Loopback Adapter and select Properties.

Note: You can configure IPv4 or IPv6 addresses or both depending on your requirements.

1. Uncheck all items except Internet Protocol Version 4 (TCP/IPv4) as shown below:

2. Ensure that Internet Protocol Version (TCP/IPv4) is selected, click Properties and configure the IP address
   to be the same as the Virtual Service address (VIP) with a subnet mask of 255.255.255.255, e.g.
   192.168.2.20/255.255.255.255 as shown below:

Note: 192.168.2.20 is an example, make sure you specify the correct VIP address.
Note: If a Real Server is included in multiple DR mode VIPs, an IP address for each VIP must be added to the Loopback Adapter.

3. Click OK then click Close to save and apply the new settings.

Step 3 of 3: Configure the strong/weak host behavior

From Windows Server 2008 onwards, Microsoft implemented a whole new network stack, and I must admit it looks like Microsoft finally have a sensible way of controlling network interfaces.

Weak and strong host behavior in Windows:

"Windows XP and Windows Server® 2003 use the weak host model for sends and receives for all IPv4 interfaces and the strong host model for sends and receives for all IPv6 interfaces. You cannot configure this behavior. The Next Generation TCP/IP stack in Windows Vista and Windows Server 2008+ supports strong host sends and receives for both IPv4 and IPv6 by default."

• You still need to configure the loopback adapter with the VIP (but you don't need to set the metric)
• You still need to disable the firewall (or enable traffic to and from the loopback)

But then you MUST use the following command line magic :

netsh interface ipv4 set interface "net" weakhostreceive=enabled
netsh interface ipv4 set interface "loopback" weakhostreceive=enabled
netsh interface ipv4 set interface "loopback" weakhostsend=enabled

Obviously first you will need to rename the specific adapters from the default of "Local Area Network Connection 1" to either "net" or "loopback" respectively i.e.

Or if you want look up the index number instead using the following command:

netsh interface ipv4 show interface

I've still got some more research to do on this but it looks like you can do all sorts of cool routing tricks in the new Windows TCP/IP stack.

Just think if we didn't have Linux to push the boundaries — Microsoft may never have been able to copy the best features so well :-)

References

The awesome advantages of Direct Routing
How can you use DSR (Direct Server Return) in AWS?
All the Load balancing methods starting with DSR...
Yahoo’s L3 Direct Server Return (DSR)
Solving the ARP problem for Linux or Windows
The LVS Project & The ARP problem

One last gotcha, if your server is set to automatically update the DNS server with its IP address then sometimes the DNS server will start giving the IP on the loopback adapter as your primary address! This is not good so disable automatic DNS entries on both the server and in the DNS management console. We'll do a separate blog on this when I have more test data.