Using the "medium_id" feature in Linux 2.2 and 2.4 Julian Anastasov , December 9, 2001 CONTENTS: 0. History 1. Introduction 2. Terminology 3. Requirements 4. Recommendations and Example 0. History 04-MAR-2002 Version 2 - a semantic change from Alexey Kuznetsov: now the medium_id values depend on the proxy_arp feature: once proxy_arp is enabled the medium_id is used. 09-DEC-2001 History started 1. Introduction The proposed feature adds the ability to differentiate the network devices by the medium they are attached to. Currently, we use this differentiation to add more control over the proxy ARP feature. By this way we try to provide feature similar to the bridging but suitable for more secure setups and easier for firewalling and QoS. And it works for NOARP devices. In short, we try to send ARP replies even for non-local IP addresses but only when we are sure that the target hosts do not listen to these broadcast probes and it is safe to connect the requestor and the target by proxying the ARP requests in both directions. The feature is used by defining medium_id for all network devices that we use for proxy ARP before enabling the proxy_arp feature for selected interfaces. Example: # eth0 and eth1 are attched to same hub, so they have same id echo 1 > /proc/sys/net/ipv4/conf/eth0/medium_id echo 1 > /proc/sys/net/ipv4/conf/eth1/medium_id # eth2 is attached to different hub echo 2 > /proc/sys/net/ipv4/conf/eth2/medium_id # ppp0 is connected to different medium echo 3 > /proc/sys/net/ipv4/conf/ppp0/medium_id # eth3 is attached to unknown medium, don't reply for its routes # and don't allow proxy_arp on it even if eth3/proxy_arp is 1 echo -1 > /proc/sys/net/ipv4/conf/eth3/medium_id # enable the proxy_arp feature after all devices are set up echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp Note that /proc/sys/net/ipv4/conf/all/medium_id is not used. The logic is (note that under indev/proxy_arp we assume the proxy_arp feature for the interface receiving the ARP probe, the value is 1 if all/proxy_arp or DEVNAME/proxy_arp is set) as follows, where indev/proxy_arp is enabled: - indev/medium_id == 0 allows proxy ARP no matter the medium_id for the target interface This is the default (old proxy_arp) behavior when proxy_arp is enabled for interface without setting its medium_id value. - indev/medium_id == -1 disables the proxy ARP because we don't know where the interface is attached. Such configuration looks reasonable when indev/proxy_arp is enabled only by setting all/proxy_arp and we tune the medium ID on interface basis. In this case, we disable proxy ARP on this interface and prohibit using it as target device from other devices with proxy_arp enabled and with valid medium ID. - in all other cases medium_id (not 0 or -1) is compared to the medium_id value of the target interface - if the target medium_id is not -1 (unknown) and both values are different then the proxy ARP is enabled. In such case we are sure the both interfaces are attached to different mediums. Of course, two mediums can use same ID > 0. By this way we disable the proxy ARP between them while at the same time it can be allowed between them and another medium with different ID. 2. Terminology The term "medium" in our case includes all network devices, from one or from many hosts, that are connected directly or indirectly/transparently (via hubs). Such examples are LANs, Point-to-Point links. In the case with the LANs we can say that all devices attached to the medium can see the broadcast packets sent from other devices attached only to the same medium. Note that one medium can be extended by using bridging or similar features. 3. Requirements The requirements to use this feature include: * We have router with two or more network devices (from any kind, one is at least an ARP device) and it is part from IPv4 subnet reachable through more than one device. These network devices are attached to different mediums. The term we use here ("subnet") is not very correct because the requestor can resolve IP address without announcing IP address from the same subnet as the target (resolved) host. * There are no other hosts from the same subnet that are attached to more than one of the mediums mentioned above. For this, the router has specific routes for each part of the subnet through these different devices, more correctly, mediums, because we can have two devices attached to same medium. * It is safe to use the same settings for other subnets over the same devices because the feature does not have such fine grained (per-route) control. All subnets on the device are affected. 4. Recommendations and Example As we already said, this feature is recommended for setups where the Linux Bridging code works just fine but due to some topology requirements we can not use it. Such setups can include ADSL modem providing large public subnet reachable through gateway from the same subnet. In such cases we can (prefer to) have the gateway connected to different medium as opposed to the internal medium we use for this public IP network. As we usually need reverse path protection for such setups we need a way the network stack to differentiate the packets received from the both mediums. This is achieved by the fact that we receive these packets on different device which is not true when the Linux Bridging code is used. The difficult part is that we have to create correct routing setup including host or network routes through the right device which is not required for the Linux Bridging. We split the subnet. As result, the proposed feature is suitable for setups where we try to classify the used devices as internal and external while at the same time we have hosts from same subnet on many of them. In such cases, the Linux Bridging code is not suitable because the network stack can not recognize from which of the bridged ports (the real network devices) the packets are received and to use this information to apply some policy. As a bonus, all other features as firewalling, QoS, etc. work by magic after we link the involved mediums by ARP. In all other cases, eg. when there are many hosts in the subnet on different mediums (it is hard to create routes for each part of this subnet) and when the network stack does not care to which of the bridged ports the packets are received we can go with the Linux Bridging. Note also that the proposed proxy ARP feature does work only for IPv4. The recommended way to use the feature is (with example settings): medium 1 10.0.0.1 -----------------------+--------------- hub 1 |eth0 +-------+-------+ | ROUTER | +----+------+---+ 10.0.0.2 eth1| |eth2 192.168.0.2/24 10.0.0.0/24 -----------------+------+----------- hub 2 medium 2 Our setup has public subnet 10.0.0.0/24 with one host (10.0.0.1 on medium 1) and all other hosts on medium 2. - register the devices, bring them up for IP ifconfig eth0 0.0.0.0 up ifconfig eth1 0.0.0.0 up ifconfig eth2 0.0.0.0 up ... - define the midium ID for all devices that will be connected to partial subnets on different mediums, for all others set medium_id to -1 echo 1 > /proc/sys/net/ipv4/conf/eth0/medium_id echo 2 > /proc/sys/net/ipv4/conf/eth1/medium_id - add the proper routes # the public subnet on medium 2 ip addr add 10.0.0.2/24 dev eth1 # Host route to gateway, on medium 1 ip route add 10.0.0.1/32 dev eth0 src 10.0.0.2 # Internal private subnet, no specific medium ID but could be 2 ip addr add 192.168.0.2/24 brd + dev eth2 # The default route ip route add default via 10.0.0.1 dev eth0 src 10.0.0.2 # Enable proxy ARP after each interface has proper medium_id echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp On other setups it is possible eth0 to be connected to 10.0.0.1 by using crossover cable. Note the main requirement: When our router has specific routes for the used subnet on different mediums (as it should be if the medium_id feature is used), the remote hosts really must be connected only to one of these mediums, exactly as specified in our routes. If we have route to some host through one medium but this host has connection to another medium that we use for this subnet then it is possible the ARP resolver to receive 2 ARP replies: one from us (the proxy server) and one from the original target host because it is connected to the same medium where the requestor is attached and broadcasts its probe. In this case, the admin cheats the ARP proxy server that the remote host is attached only to one of the mediums and the results are fatal. Also note that the IP addresses in Linux are not device property but a host property. In other words, don't try to connect the remote hosts to two or more mediums with the excuse that all IP addresses from the used subnet are configured on devices attached only to one meduim. That simply does not work, at least, it depends on the used routes and on the filtering of the ARP probes. Linux replies no matter on what device is configured the IP address and the result will be 2 ARP replies for one probe.