Search Ali's Blog

Monday, September 20, 2010

The ARP Flux Problem

If you spent a lot of time trying to understand why a network interface in Linux fakes ARP replies on behalf of another interface, you may be facing the ARP flux effect! Yes, this is how it's called, and yes, you spend many hours debugging problems due to this weird behavior.

Surprisingly, this is a known behavior of Liux ARP module, when a a machine has multiple network interfaces, as stated in O'Reilly Book: Understanding Linux Network Internals:



The solution of the flux problem is through altering the system sysctl parameters, few websites (see references below) suggest how to make the suitable changes.

To make this process easier, you can use this script to set the right values into the procfs files.
To make the changes permanent (kept among reboots): the following can be added to your /etc/sysctl.conf file:
sysctl -w net.ipv4.conf.all.arp_ignore=1
sysctl -w net.ipv4.conf.all.arp_announce=2

Tip:
To flush the ARP table in Linux, use this helper script.

References:
http://wiki.openvz.org/Multiple_network_interfaces_and_ARP_flux
http://www.inlab.de/balanceng/faq.html
http://linux-ip.net/html/ether-arp.html

2 comments:

Amir Watad said...

Does it behave like this for a reason? why isn't it get fixed in linux?

Ali Ayoub said...

In some cases this behavior can be good, for example if you have multiple interfaces connected to the same subnet and you do not care what interface answers any incoming packet then this gives you some redundancy. However, if (a) you do care what interface should answer (for example if one interface is dedicated for some specific performance application) or (b) multiple interfaces are in different subnets. Then the ARP flux behavior becomes problematic.

I still think that ARP flux should be disabled by default, but apparently this is kept for historical reasons as ARP module design considers the IP address to belong to the host rather than interface.