The netstat utility summarizes a variety of characteristics of the networking stack. With netstat you can learn a number of important things. If no other type of data is requested it will report on the state of all active sockets. You can however request the routing table, masquerading table, network interface statistics, and network stack statistics [58].
One of the most common uses of the netstat utility is to determine the state of sockets on a machine. There are many questions that netstat can answer with the right set of options. Here's a list of some of the things different things we can learn.
which services are listening on which sockets
what process (and controlling PID) is listening on a given socket
whether data is waiting to be read on a socket
what connections are currently established to which sockets
By invoking netstat without any options, you are asking for a list of all currently open connections to and from the networking stack on the local machine. This means IP network connections, unix domain sockets, IPX sockets and Appletalk sockets among others. Naturally, we'll skip over the non-IP sockets since this is about IP networking with linux.
Assume the --inet switch in all cases below unless we are examining a particular higher layer protocol (e.g., TCP with the --tcp switch or UDP with --udp switch.
A convenient feature of netstat is its ability to differentiate between two different sorts of name lookup. Normally the -n specifies no name lookup, but this is ambiguous when there are hostnames, port names, and user names. Fortunately, netstat offers the following options to differentiate the different forms of lookup and suppress only the [un-]desired lookup.
--numeric-hosts
--numeric-ports
--numeric-users
The option -n (my favorite), suppress all hostname, port name and username lookup, and is a synonym for --numeric. I'll reiterate that hostnames and DNS in particular can be confusing, or worse, misleading when trying to diagnose or debug a networking related issue, so it is wise to suppress hostname lookups in these sorts of situations.
In Example G.11 we will look at netstat's numeric output and then we'll invoke the same command but suppress the host lookups. Though the output is almost the same, a particular situation might call for one or the other invocation.
Example G.11. Displaying IP socket status with netstat
[root@morgan]# netstat --inet -n
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 192 192.168.98.82:22 192.168.99.35:40991 ESTABLISHED
tcp 0 0 192.168.98.82:42929 192.168.100.17:993 ESTABLISHED
tcp 96 0 127.0.0.1:40863 127.0.0.1:6010 ESTABLISHED
tcp 0 0 127.0.0.1:6010 127.0.0.1:40863 ESTABLISHED
tcp 0 0 127.0.0.1:38502 127.0.0.1:6010 ESTABLISHED
tcp 0 0 127.0.0.1:6010 127.0.0.1:38502 ESTABLISHED
tcp 0 0 192.168.98.82:53733 209.10.26.51:80 SYN_SENT
tcp 0 0 192.168.98.82:44468 192.168.100.17:993 ESTABLISHED
tcp 0 0 192.168.98.82:44320 192.168.100.17:139 TIME_WAIT
[root@morgan]# netstat --inet --numeric-hosts
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 192.168.98.82:ssh 192.168.99.35:40991 ESTABLISHED
tcp 0 0 192.168.98.82:42929 192.168.100.17:imaps ESTABLISHED
tcp 0 0 127.0.0.1:40863 127.0.0.:x11-ssh-offset ESTABLISHED
tcp 0 0 127.0.0.:x11-ssh-offset 127.0.0.1:40863 ESTABLISHED
tcp 0 0 127.0.0.1:38502 127.0.0.:x11-ssh-offset ESTABLISHED
tcp 0 0 127.0.0.:x11-ssh-offset 127.0.0.1:38502 ESTABLISHED
tcp 0 0 192.168.98.82:53733 209.10.26.51:http SYN_SENT
tcp 0 0 192.168.98.82:44468 192.168.100.17:imaps ESTABLISHED
tcp 0 0 192.168.98.82:44320 192.168.100:netbios-ssn TIME_WAIT
|
Each line represents a either the sending or receiving half of a connection. In the above output on morgan it appears that there are no connections other than TCP connections. If you are very familiar with TCP ports and the service associated with that port, then the first format will suffice in most cases. A possibly misleading aspect of the latter output is visible in the connections to and from localhost and the final line. netstat abbreviates the IP endpoint in order to reproduce the entire string retrieved from the port lookup (in /etc/services). Also interestingly, this line conveys to us (in the first output) that the kernel is waiting for the remote endpoint to acknowledge the 192 bytes which are still in the Send-Q buffer.
The first line describes a TCP connection to the IP locally hosted on morgan's Ethernet interface. The connection was initiated from an ephemeral port (40991) on tristan to a service running on port 22. The service normally running on this well-known port is sshd, so we can conclude that somebody on tristan has connected to the morgan's ssh server. The second line describes a TCP session open to port 993 on isolde, which probably means that the user on morgan has an open connection to an IMAP over SSL server.
The third through the sixth lines can be understood in pairs. By examining the source and destination IP and port pairs, we can see that two different TCP sessions have been established with the source and destination address of 127.0.0.1. For an administrator to publish services on localhost is not at all uncommon. This makes the service harder to abuse from the network. In this case, when we allow the service lookup, the port in question (6010) appears to be used to tunnel forwarded X applications over ssh.
The next line is the first TCP session in our output which is not in a state of ESTABLISHED. Refer to Table G.1 for a full list of the possible values of the State field in the netstat output. The state SYN_SENT means that an application has made arequest for a TCP session, but has not yet received the return SYN+ACK packet.
The final line of our netstatoutput shows a connection in the TIME_WAIT state, which means that the TCP sessions have been terminated, but the kernel is waiting for any packets which may still be left on the network for this session. It is not at all abnormal for sockets to be in a TIME_WAIT state for a short period of time after a TCP session has ended.
If we needed to know exactly which application owned a particular network connection, we would use the -p | --program switch which gives us the PID and process name of the owner process. If we want to see the unix user and the PID and process we'll add the -e | --extend switch.
Example G.12. Displaying IP socket status details with netstat
[root@masq-gw]# netstat -p -e --inet --numeric-hosts
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name
tcp 0 0 192.168.100.254:ssh 192.168.100.17:49796 ESTABLISHED root 25453 6326/sshd
tcp 0 240 192.168.99.254:ssh 192.168.99.35:42948 ESTABLISHED root 171748 31535/sshd
|
There doesn't appear to be a large number of connections to and from the masq-gw host. The two sessions are initiated to the sshd running on port 22, and the process which owns each socket is a root process.
Table G.1. Possible Session States in netstat output
| State | Description |
|---|---|
| LISTEN | accepting connections |
| ESTABLISHED | connection up and passing data |
| SYN_SENT | TCP; session has been requested by us; waiting for reply from remote endpoint |
| SYN_RECV | TCP; session has been requested by a remote endpoint for a socket on which we were listening |
| LAST_ACK | TCP; our socket is closed; remote endpoint has also shut down; we are waiting for a final acknowledgement |
| CLOSE_WAIT | TCP; remote endpoint has shut down; the kernel is waiting for the application to close the socket |
| TIME_WAIT | TCP; socket is waiting after closing for any packets left on the network |
| CLOSED | socket is not being used (FIXME. What does mean?) |
| CLOSING | TCP; our socket is shut down; remote endpoint is shut down; not all data has been sent |
| FIN_WAIT1 | TCP; our socket has closed; we are in the process of tearing down the connection |
| FIN_WAIT2 | TCP; the connection has been closed; our socket is waiting for the remote endpoint to shut down |
One of the most common uses of netstat, especially in cross-platform environments is the reporting of the main routing table. On many platforms, netstat -rn is the preferred method of displaying routing information, although linux provides at least two alternatives to this: route and ip route show.
Example G.13. Displaying the main routing table with netstat
[root@morgan]# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.98.0 0.0.0.0 255.255.255.0 U 40 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 40 0 0 lo
0.0.0.0 192.168.98.254 0.0.0.0 UG 40 0 0 eth0
|
This output should look familiar. The routing cache itself may not be as familiar to most, but can also be displayed with netstat. The ouput below is exactly the same as the ouput from route -enC. Refer also to Example D.3.
Example G.14. Displaying the routing cache with netstat
[root@tristan]# netstat -rnC
Kernel IP routing cache
Source Destination Gateway Flags MSS Window irtt Iface
194.52.197.133 192.168.99.35 192.168.99.35 l 40 0 0 lo
192.168.99.35 194.52.197.133 192.168.99.254 1500 0 29 eth0
192.168.99.35 192.168.99.254 192.168.99.254 1500 0 0 eth0
192.168.99.254 192.168.99.35 192.168.99.35 il 40 0 0 lo
192.168.99.35 192.168.99.35 192.168.99.35 l 16436 0 0 lo
192.168.99.35 194.52.197.133 192.168.99.254 1500 0 0 eth0
192.168.99.35 192.168.99.254 192.168.99.254 1500 0 0 eth0
|
Consult Section D.1.1, “Displaying the routing table with route” for more detail on reading and interpreting the data in this output. Because this is simply another way of reporting the routing table information, we'll skip over any detailed description.
netstat -i summarizes interface statistics in a terse format. This format
OK! This is strange. netstat -ie looks exactly like ifconfig output. That's weird!
For machines which perform masquerading, typically dual-homed packet-filtering firewalls like masq-gw a tool to list the current state of the masquerading table is convenient.
Each masqueraded connection can be described by a tuple of six pieces of data: the source IP and source port, the destination IP and destination port, and the (usually implicit) locally hosted IP and a local port.
FIXME; this command seems to fail on all of the iptables boxen, even if I'm using the -j MASQUERADE target. I can use it successfully on ipchains boxen. Anybody have any ideas or explanation here?
[58] Additionally, netstat can display multicast information with the --group switch. I have zero experience here. Anybody with experience want to write about this?