Back to my home page
1. Descriptiondummynet is a flexible tool originally designed
for testing networking protocols, and since then (mis)used for bandwidth
It simulates/enforces queue and bandwidth limitations, delays, packet losses,
and multipath effects. It also implements a variant of Weighted Fair Queueing
called WF2Q+. It can be used on user's workstations, or on FreeBSD machines
acting as routers or bridges.
Just to get the idea of what you can do with dummynet, e.g. by using dummynet
on your workstation, or putting a PC with two ethernet cards between your
network and your router and booting from the floppy-image
below, here are a few examples :
These rules limit the total ICMP traffic (inbound+outbound) to 50Kbit/s
ipfw add pipe 1 icmp from any to any
ipfw pipe 1 config bw 50Kbit/s queue 10
These rules limit inbound traffic to 300Kbit/s for each host on your network
ipfw add pipe 2 ip from any to 10.1.2.0/24
ipfw pipe 2 config bw 300Kbit/s queue 20 mask dst-ip 0x000000ff
If you want all machines to share evenly a single link, you should use
ipfw add queue 1 ip from any to 10.1.2.0/24
ipfw queue 1 config weight 5 pipe 2 mask dst-ip 0x000000ff
ipfw pipe 2 config bw 300Kbit/s
And these rules simulate an ADSL link to the moon:
ipfw add pipe 3 ip from any to any out
ipfw add pipe 4 ip from any to any in
ipfw pipe 3 config bw 128Kbit/s queue 10 delay 1000ms
ipfw pipe 4 config bw 640Kbit/s queue 30 delay 1000ms
dummynet works by intercepting packets (selected by ipfw rules -
ipfw is one of the FreeBSD firewalls) in their way through the protocol stack,
and passing them through one or more objects called queues and
pipes, which simulate the effects of bandwidth limitations, propagation
delays, bounded-size queues, packet losses, multipath. Pipes are fixed-bandwidth
channels. Queues represent instead queues of packets, associated with a weight,
which share the bandwidth of the pipe they are connected to proportionally to
Each pipe and queue can be configured separately, so you can apply different
limitations/delays to different traffic according to the ipfw rules (e.g.
selecting on protocols, addresses and ports ranges, interfaces, etc.). Pipes and
queues can be created dynamically, so using a single set of rules you can apply
independent limitations to all hosts in a subnet, or to all types of traffic,
etc. You can also configure the system to build cascades of pipes, so you can
simulate networks with multiple links and paths between source(s) and
2. Performance, status and availabilityUnlike other traffic shaping
packages which run in userland, dummynet has a very little overhead, as all
processing is done within the kernel. There is no data copying involved to move
packets through pipes, just a bit of pointer shuffling, and the implementation
is able to handle thousands of pipes with O(log N) cost.
The WFQ variant we implement, called WF2Q+, has a complexity which is O(log
N) in the number of active flows, so again it is able to handle efficiently
thousands of flows. dummynet is part of FreeBSD since Sept.1998. It has been recently
(Jan.2000 and June 2000) rewritten, so the most recent, feature-rich and robust
versions are in FreeBSD 3.4-STABLE and newer releases.
You don't need to install FreeBSD on your hard disk to use it, as below you will
find a bootable single-floppy version of FreeBSD which includes dummynet,
bridging, and a lot of other goodies.
Dummynet is being heavily used by lots of people, and the code seems to be
extremely stable and robust, especially in the 3.4-STABLE version and above. Bug
fixes are generally applied to the FreeBSD source tree and are available from
the CVS tree or in
newer snapshot/releases of FreeBSD. From time to time i update the floppy image
on this site as well.
3. SupportIf you have found some bug, please report it to me by
email, but don't forget to include information on which version of FreeBSD and
dummynet you are using, your rules (ipfw show; ipfw pipe show), your
configuration (bridge or router) etc.
If you have a simple question, again
just email me and i generally try to reply as soon as possible. Again, please
For more complex things (like "i have no time to learn how to
use it, i just want this work done"), or customizations and additions of new
features to dummynet/ipfw, I am available (through my department) for doing
support on a contract basis.
Email email@example.com for discussing details.
This said, FreeBSD users should be able to use dummynet without the need for
The relevant manpages (ipfw(8), dummynet(4), bridge(4)) are
a great source of information, so please read updated version of them before
You can also try posting on the various FreeBSD mailing
lists or newsgroups, they are usually a very good source of information.
4. Using dummynetDummynet is entirely controlled by the ipfw
commands and a set of sysctl variables.
4.1 Basic ipfw commandsThe basic structure of ipfw commands is
ipfw add [N] [prob X] action PROTO from SRC to DST [options]
where N is the rule number ;
X is a number between 0 and 1 that, when
present, indicates the probability of getting a match on this rule if all other
fields are correct. The default is deterministic match;
one of the actions executed on a match, which can be any of allow, deny,
skipto N, pipe N and others. To send a packet to a dummynet pipe, we have
to use pipe N; PROTO is the protocol type we want to match (IP, TCP,
SRC and DST are address specifier (we can use addresses with
netmasks and optionally followed by ports or port ranges);
can be used to restrict the attention to packets coming from/to specific
interfaces, or carrying some TCP flags or ICMP options, or bridged, etc.
4.2 Sysctl variablesThe following are the main sysctl variables to
control the behaviour of ipfw, bridging and dummynet:
Controlling ipfwThe firewall is mostly controlled by ipfw, and
the sysctl variables only serve to give global configuration and default
enables firewall in the IP stack
Forces a single pass through the firewall. If set to 0,
packets coming out of a pipe will be reinjected into the
firewall starting with the rule after the matching one.
NOTE: there is always one pass for bridged packets.
net.inet.ip.fw.dyn_buckets: 256 (readonly)
Current hash table size used for dynamic rules.
Desired hash table size used for dynamic rules.
Current number of dynamic rules. (readonly)
Max number of dynamic rules. If you exceed this limit, you will
have to wait for a rule to expire before being able to create
a new one.
Lifetime (in seconds) for various types of dynamic rules.
Controlling dummynetAlso dummynet is mostly controlled by
ipfw, with the sysctl variables serving mostly for default parameters.
Size of hash table for dynamic pipes.
Delete dynamic pipes when they become empty.
Max ratio between number of dynamic queues and hash buckets.
When you exceed (max_chain_len*buckets) queues on a pipe,
packets not matching any of these will be all put into the
same default queue.
Controlling bridgingBridging is almost exclusively controlled by sysctl
set of interfaces for which bridging is enabled, and cluster
they belong to.
enable ipfw for bridging.
4.3 Pipe and queue configurationThe following ipfw commands
control dummynet pipes
parameters can be configured for a pipe, adding the command in the pipe
- ipfw pipe NN config ...
This command is used to create or
reconfigure a pipe. NN is the numeric identifier (between 1 and 65535) of the
pipe. Issuing multiple time the configuration command results in the pipe
- ipfw [-s field] pipe [NN] show
This command shows the
parameters of a pipe. If the pipe is a dynamic one (see mask
parameter), then all dynamic pipes created from this one are listed. The list
can be very very long. The -s option allows you to sort the listing on one of
the four counters associated to the pipe.
- ipfw pipe NN delete Destroys a single pipe. Remember that packets
sent to a non-existing pipe are silently dropped.
- ipfw pipe flush Destroys all pipes.
To use WF2Q+, packets must be passed to queues which in turn must
be connected to a pipe.
- Bandwidth: bw NNunit
NN is the bandwidth assigned to
the pipe, unit (which must follow the number with no intervening spaces) can
be any of bit/s Kbit/s Mbit/s Byte/s KByte/s MByte/s or non-ambiguous
A bandwidth of 0 (or no bandwidth) results in no bandwidth
limitations (hence, no queues will ever build up).
- Queue size: queue NN [unit]
Sets the queue size, in
slots if only NN is specified, otherwise in Bytes or KBytes. When there is no
room in the queue, packets are dropped. The default queue size is 50
The combination of bandwidth and queue size influence the queueing
delay. Be careful when using low bandwidths not to use too large queues, or
you might end up with several seconds of queueing delay.
Also be careful
when you specify the queue size in packets: if you run tests over the loopback
interface, a packet can be very large, e.g. 16KB, again resulting in huge
- Delay: delay NN ms
Sets the propagation delay of the
pipe, in milliseconds. Note that the queueing delay component is independent
of the propagation delay. Also note that all delays are approximated with a
granularity of 1/HZ seconds (HZ is typically 100, but we suggest using HZ=1000
and maybe even larger values).
- Random Packet Loss: plr X
X is a floating point number
between 0 and 1 which causes packets to be dropped at random. This is done
generally to simulate lossy links. The default is 0, or no loss.
- Dynamic queue creation: mask ...
It is possible to
associate a mask to a pipe so that bandwidth and queue limitations are
enforced separately for packets belonging to different flows.
The mask command lets you specify which parts of the following
fields contribute to identify a flow:
[proto N] [src-ip N] [dst-ip N] [src-port N] [dst-port N]
where N is a bitmask where significant bits are set to 1. You can
specify one or more masks, or the all keyword to mean that all fields
are fully significant.
The default (when no mask are specified) is to
ignore all fields, so that all packets are considered to belong to the same
Whenever a new flow is encountered, a new queue (with the specified
bandwidth and queue size) is created.
WARNING!!! the number of dynamic queues that can be created in this
way can become very large. They are accessed through a hash table, whose size
you can define using the buckets NN specifier after the mask command.
The following ipfw commands control dummynet pipes
parameters can be configured for a queue, adding the command in the queue
- ipfw queue NN config ...
This command is used to create or
reconfigure a queue. NN is the numeric identifier (between 1 and 65535) of the
queue. Issuing multiple time the configuration command results in the queue
- ipfw queue NN delete Destroys a single queue. Remember that
packets sent to a non-existing queue are silently dropped.
- ipfw queue flush Destroys all queues.
- Pipe: pipe NN
NN is the identifier of the pipe used for
- Weight: weight NN
NN is the weight (1..100, default 1)
associated to the queue.
- Per-Flow queueing: mask ...
The syntax is the same as
for pipes. However, all queues created dynamically will share the parent
pipe's bandwidth according to the weight.
- Queue size, Random Packet Loss:
Same as for pipes.
5. Using dummynet for testing protocolsDummynet was originally
created to test network protocols and applications, possibly even on a
standalone system. As a consequence, some of its features such as delay
emulation, random loss etc. are esplitictly designed for that purpose.
There are a few things you should take in mind when doing such tests, to
avoid getting incorrect results. They are all obvious things, still it is
better to have them in mind.
- Choosing a reasonable buffer size.
As said earlier, packet can
be subject to a delay which is proportional to the total queue size (in
bytes), and inversely proportional to the bandwidth. At low bandwidths, this
queueing delays can be extremely high, especially if the queue size is
defined in terms of packets and packets are large. The default queue size is
almost certainly too large for most purposes, and it is often preferable to
define the queue size in terms of bytes rather than packets.
- Half-duplex vs. Full-duplex channels.
With the exception of
shared-medium networks such as the ethernet, most links that you want to
simulate for your experiments are full duplex. As such, the proper
configuration is the following:
ipfw add pipe 1 ip from A to B
ipfw add pipe 2 ip from B to A
ipfw pipe 1 config ...
ipfw pipe 2 config ...
Should you really need to mode a half duplex network, then you can use
the following sequence. But think twice before you do so, as it is often a
ipfw add pipe 3 ip from A to B
ipfw add pipe 3 ip from B to A
ipfw pipe 3 config ...
- Interactions between bridging and multicast
You can use ipfw
(and dummynet) in a bridge by setting some sysctl variables:
sysctl -w net.link.ether.bridge=1
sysctl -w net.link.ether.bridge_ipfw=1
and then specify your firewall configuration.
Be careful when you
run experiment involving multicast traffic through a dummynet-enabled
bridge. Unless you set the rules right, multicast traffic in a bridge goes
through the firewall code twice: once during forwarding at level 2, once
when the packet is passed to the local IP stack of the bridge.
Starting from Feb.2000, there are to avoid this problem. One involves a
sysctl -w net.inet.ip.fw.enable=0
which avoids that the firewall is invoked at the ip level. Otherwise,
you can use the bridged specifier in your ruleset to match only
ipfw add pipe 1 ip from any to any bridged
- Running over the loopback interface.
Dummynet was originally
designed for running experiments on a standalone machine. The loopback
interface lets you run senders and receivers on the same machine, but you
should remember a few things:
- TCP defaults.
Be very careful when using TCP, especially
between processes running on the same machine, or on the same
Apart from the MTU issue mentioned earlier, at least on FreeBSD,
TCP starts with a full window when the remote endpoint is on the same subnet
as one of the local addresses. You need a simple fix in the source
(tcp_input.c i believe) to fix this behaviour in FreeBSD 3.x, whereas
FreeBSD 4.x has sysctl variable(s) to set the initial window.
when you do experiments on configuration with a large delay-bandwidth
product, remember that many applications use the default window size which
is small, something like 16KB. You might end up not using the full bandwidth
just because your data transfer is window-limited.
5.1 Simulating multipathOne nice feature of the new version of
dummynet is the ability to simulate multiple paths between sender and
receiver. This is done using probabilistic match, e.g.:
ipfw add prob 0.33 pipe 1 ip from A to B
ipfw add prob 0.5 pipe 2 ip from A to B
ipfw add pipe 3 ip from A to B
ipfw pipe 1 config ...
ipfw pipe 2 config ...
ipfw pipe 3 config ...
Given the right packet, the first rule will match with probability 1/3;
in the remaining 2/3 of occurrence we move to the second rule, which will
match with prob 1/2 (so overall 1/2*1/3 = 1/3), and the remaining 1/3 of
occurrence will move to the third rule, which has a deterministic match. We
can then configure the three pipes as desired to emulate phenomena such as
packet reordering etc.
6 Related linksHere i collect some info on how to do various
ipfw-related things. Most of this is just URLs collected from the mailing list
so the reliability of the info might be different (for good or bad) from what
is in this page.
A PicoBSD floppyTo conclude... if you want to try dummynet, here
is a bootable floppy image of a system with FreeBSD, bridging, ipfw, dummynet,
natd, ppp, drivers for a few interfaces, and accessible via telnet.
To setup this system, download the 1.44MB image, pico.000608.bin
and copy it to a floppy using dd under FreeBSD, or rawrite
Then put the floppy into a machine with hopefully at least one interface,
and wait for it to boot. When the system comes up, login as root, password
"setup", and you can play with bridging, ipfw and dummynet using the above
Dipartimento di Ingegneria dell'Informazione -- Univ. di
via Diotisalvi 2 -- 56126 PISA
tel. +39-050-568533 Fax