|   Search  |   Site Map  |   Help  |   Contacts

Papers TCP Rate-Halving NIMI Autotuning Projects TCP Rate-Halving NIMI Autotuning SACK/FACK Technology    Integration Software TCP Implementations TReno Traceroute Windowed Ping Websites TCP Performance    Debugging Performance    Tuning TCP Friendly Related Projects NLANR NCNE Engineering    Services NCNE GigaPop PSC LBNL NRG Miscellaneous Staff Help Search Web Feedback

Enabling High Performance Data Transfers System Specific Notes for System Administrators (and Privileged Users) Please Note: This information was originally compiled by Jamshid Mahdavi (Thanks a bunch, Jamshid!). Many, many people have helped us in compiling the information on this website. We want to thank all of them for their help in sending us updates, and encourage people to continue to report any errors or additions to us at nettune@ncne.org so that the information herein will be as up-to-date as possible. We are especially interested in information on the latest OS releases. If the instructions here apply to newer releases please let us know so we can update the version information. Introduction Overview Bandwidth*Delay Products (BDP) Buffers Computing the BDP High Performance Networking Options TCP Selective Acknowledgments (SACK, RFC2018) Large Windows (RFC1323) Maximum Buffer Sizes on the host Application Buffers Path MTU Using Web Based Network Diagnostic Servers SYN Test Server ORNL Bandwidth Tester ANL Network Diagnostic Tool Support for various operating systems Detailed Procedures for System tuning and Raising network limits under BSD/OS 2.1 & 3.0 on CRI systems under Unicos 8.0 on (Compaq) DEC Alpha systems under Digital Unix 3.2c under FreeBSD 2.1.5 under HP-UX 9.X, 10.x & 11.X on IBM RS/6000 systems under AIX 3.2 or 4.1 on IBM MVS systems under the Interlink TCP Stack on Linux systems for 2.4.00 or greater on Macintosh Open Transport systems under NetBSD under FTP Software (NetManage) OnNet 4.0 for Windows 95/98 on SGI systems under IRIX 6.5 on Sun Solaris Systems Miscellaneous information about Windows NT on Microsoft Windows 98 systems on Microsoft Windows 2000 and Windows XP systems

In the following section titled "Overview" we will briefly explain the issues, and also define some terms. A later section under "Detailed Procedures" provides some step by step directions on making the necessary changes under various operating systems.

Overview

Bandwidth*Delay Products (BDP)

   1,000,000,000 bits    1 byte   70 seconds
   ------------------- * ------ * ---------- = 8,750,000 bytes = 8.75 MBytes
   1 second              8 bits   1,000

The next section presents a brief overview of the high performance options. Specific details on how to enable these options in various operating systems is provided in a later section.

High Performance Networking Options

1. TCP Selective Acknowledgments (SACK, RFC2018): SACKs allow a receiver to acknowledge non-consecutive data. This is particularly helpful on paths with a large Bandwidth-Delay-Product (BDP). While SACK is now supported by most operating systems, it may have to be explicitly turned on by the system administrator.

   Additional information on commercial and experimental implementations of SACK is available at http://www.psc.edu/networking/all_sack.html.

2. Large Windows (RFC1323): Without the support of this TCP enhancement, the buffer sizes that can be used by the application is limited to 64K Bytes. As we have seen in the BDP section above, this would be inadequate for today's high speed WANs.

   On most systems, RFC1323 extensions are included but may require the system administrator to explicitly turn them on.

3. Maximum Buffer Sizes on the host: Typically operating systems limit the amount of memory that can be used by an application for buffering network data. The host system must be configured to support large enough socket buffers for reading and writing data to the network. Typical Unix systems include a default maximum value for the socket buffer size between 128 kB and 1 MB. For many paths, this is not enough, and must be increased.

   Please note that without RFC1323 "Large Windows" indicated above, TCP/IP does not allow applications to buffer more the 64 kB in the network, irrespective of the maximum buffer size configured.

4. Default Buffer Sizes: The "Maximum Buffer Size" mentioned above sets a maximum limit for the buffers (as you may have guessed by the name!). In addition to this, most operating systems have a system-wide "default buffer size" that is configured in. Unless an application explicitly requests a specific buffer size, it gets a buffer of the default size. Most system administrators set this value such that it is appropriate for a LAN, but would not necessarily be sufficient for WAN with a large BDP.

   System administrators usually have to make a judicious choice for default value and maximum value for the buffers.

5. Application Buffers: If the default buffer size is not large enough, the application must set its send and receive socket buffer sizes (at both ends) to at least the BDP of the link. Some network applications support options for the user to set the socket buffer size (for example, Cray UNICOS FTP); many do not. There are several modified versions of applications available which support large socket buffer sizes.
   • NLANR/DAST has created an FTP client which automatically sets the socket buffer size to the measure bandwidth*delay product for the path. This client can be found at http://dast.nlanr.net/Projects/Autobuf_v1.0/autotcp.html
   • Retrieve an example of a user tunable version of WU-FTP and NCFTP from NCAR.
   • NLANR/NCNE maintains a tool repository which includes application enhancements for several versions of FTP and rsh. Also included on this site is the nettune library for performing such enhancements yourself.

   Alternatively, the system-wide default socket buffer size can be raised, causing all applications to utilize large socket buffers. This is not generally recommended, as many network applications then consume system memory which they do not require.

   New: The best solution would be for the operating system to automatically tune socket buffers to the appropriate size. Linux users may want to investigate Web100, which includes autotuning and TCP instrumentation to enable better TCP diagnosis. This follows earlier work by Jeff Semke at PSC where he developed an experimental Autotuning Implementation for NetBSD. In the future, we hope to see such automatic tuning as a part of all TCP implementations, making this entire website obsolete.

   For socket applications, the programmer can choose the socket buffer sizes using a setsockopt() system call. A Detailed Users Guide describing how to set socket buffer sizes within socket based applications has been put together by Von Welch at NCSA.

6. Path MTU The host system must use the largest possible MTU for the path. This may require enabling Path MTU Discovery (RFC1191). Since RFC1191 is flawed this is never enabled by default and need to be explicitly enabled by the system administrator. Watch for pMTU blackholes. If Path MTU Discovery is unavailable or undesired, it is sometimes possible to trick the system into using large packets, but this may have undesirable side effects.

   The Path MTU Discovery server (described in a little more detail in the next section) may be useful in checking out the largest MTU supported by some paths:

   http://www.ncne.org/jumbogram/mtu_discovery.php

• SYN Test Server: Often users would like a quick way to check if the high performance networking options discussed above (SACK etc.) are turned on. This is a Web100 based server that allows any client on the internet to do this quickly and easily. It can be accessed either from a text window or from a web browser. In text mode, one can simply type:

             telnet syntest.psc.edu 7960
  

  Alternatively, one can point a web browser on the host to the following URL: http://syntest.psc.edu:7961

• ORNL TCP Bandwidth Tester: Sometimes users would like to run a short network test to a remote machine to see how well connected they are to the outside world. This is a web based java server that does a couple of short network tests and measures bandwidth between the host and the server. Simply visit the following URL: http://www.epm.ornl.gov/~dunigan/java/misc/tcpbw.html

• ANL Network Configuration Tester: This is a web based java server that does a couple of short network tests and provides diagnostic information about the host. This server is particularly helpful in troubleshooting duplex mismatch problems.
  Simply visit the following URL: http://miranda.ctd.anl.gov:7123/
  Detailed explanation about capabilities of tool can be found at the web site.

• Path MTU Discovery Server: This is a web based server you can use to determine the largest MTU supported between the server at Pittsburgh Supercomputing Center and the host you are interested in:

  http://www.ncne.org/jumbogram/mtu_discovery.php

  This tool shows the path and the largest MTU supported on that path from a server at the Pittsburgh Supercomputing Center to the host in question.

Procedure for raising network limits under BSD/OS 2.1 and 3.0 (BSDi)

Procedure for raising network limits under CRI systems under Unicos 8.0

% /etc/netvar
Network configuration variables
        tcp send space is 32678
        tcp recv space is 32678
        tcp time to live is 60
        tcp keepalive delay is 14400
        udp send space is 65536
        udp recv space is 68096
        udp time to live is 60
        ipforwarding is on
        ipsendredirects is on
        subnetsarelocal is on
        dynamic MTU discovery is on
        adminstrator mtu override is on
        maximum number of allocated sockets is 3750
        maximum socket buffer space is 409600
        operator message delay interval is 5
        per-session sockbuf space limit is 0

• dynamic MTU discovery: This is "off" by default and should be changed to "on".

• maximum socket buffer space: This should be set to the desired maximum socket buffer size (in bytes).

• tcp send space, tcp recv space: These are the default buffer sizes used by applications. These should be changed with caution.

Procedure for raising network limits on (Compaq) DEC Alpha systems under Digital Unix 3.2c

• By default, the maximum allowable socket buffer size on this operating system is 128kB.

• In order to raise this maximum, you must increase the kernel variable sb_max. In order to do this, run the following commands as root:

  # dbx -k /vmunix
  (dbx) assign sb_max = (u_long) 524288
  (dbx) patch sb_max = (u_long) 524288
  

  In this example, sb_max is increased to 512kB. The first command changes the variable for the running system, and the second command patches the kernel so it will continue to use the new value, even after rebooting the system. Note, however, that reinstalling (overwriting) the kernel will undo this change.

• The Digital Unix manuals also recommend increasing mbclusters to at least 832.

• Standard applications do not have a mechanism for setting the socket buffer size to anything but the default. However, you can change the kernel default by modifying the kernel variables (tcp_sendspace, tcp_recvspace)

Procedure for raising network limits on (Compaq) DEC Alpha systems under Digital Unix 4.0

• Under version 4.0 of Digital Unix, many variables can now be tuned with the sysconfig command. Some (but not all!) of the relevant variables from sysconfig are shown here:

  
  	    % /sbin/sysconfig -q inet
  	    inet:
  	    tcp_sendspace = 32768
  	    tcp_recvspace = 32768
  	    tcp_keepidle = 14400
  	    tcp_keepintvl = 150
  	    tcp_keepinit = 150
  	    tcp_keepcnt = 8
  	    tcp_ttl = 60
  	    tcp_mssdflt = 536
  	    tcp_rttdflt = 3
  	    tcp_dont_winscale = 0
  	    tcpnodelack = 0
  	    tcptwreorder = 1
  	    udp_sendspace = 9216
  	    udp_recvspace = 41600
  	    udpcksum = 1
  	    udp_ttl = 30
  	    pmtu_enabled = 1
  	    pmtu_rt_check_intvl = 20
  	    pmtu_decrease_intvl = 1200
  	    pmtu_increase_intvl = 240
  	    ...
  
  	    % /sbin/sysconfig -q socket
  	    socket:
  	    sominconn = 0
  	    somaxconn = 1024
  	    sb_max = 131072
  
  

  To make a change (for example):

              # /sbin/sysconfig -r inet tcp_sendspace 65536
              # /sbin/sysconfig -r inet tcp_recvspace 65536
  

• Specific advice for tuning (Compaq) Digital UNIX systems (for both V4.0 releases and many of the V3.2x releases) may be found at http://www.unix.digital.com/internet/tuning.htm

  This document contains information on other important parameters (not just the ones directly associated with the socket, IP, and TCP layers) and gives instructions on how to modify things. It also includes important patch information, and is updated every few months.

Procedure for raising network limits under FreeBSD 2.1.5

You can't modify the maximum socket buffer size in FreeBSD 2.1.0-RELEASE, but in 2.2-CURRENT you can use

	sysctl -w kern.maxsockbuf=524288

	net.inet.tcp.sendspace
	net.inet.tcp.recvspace
	net.inet.udp.recvspace

Procedure for raising network limits under HPUX 9.X

There are patches for 9.05 and 9.07 that provide 1323 support. To enable it, one must poke the kernel variables tcp_dont_tsecho and tcp_dont_winscale to 0 with adb (the patch includes a script, but I don't recall the patch number).

Without the 9.05/9.07 patch, the maximum socket buffer buffer size is somewhere around 58254 bytes. With the patch it is somewhere around 1MB (there is a small chance it is as much as 4MB).

The FTP provided with the up to date patches should offer an option to change the socket buffer size. The default socket buffer size for this could be 32KB or 56KB.

There is no support for SACK in 9.X.

Procedure for raising network limits under HPUX 10.X

Up through 10.20, RFC 1323 support is like the 9.05 patch, except the maximum socket buffer size is somewhere between 240 and 256KB. In other words, you need to do the same adb "pokes" as described above.

10.30 does not require adb "pokes" to enable RFC1323. 10.30 also replaces nettune with ndd. The 10.X default TCP socket buffer size is 32768, the default UDP remains unchanged from 9.X. Both can be tweaked with nettune.

FTP should be as it is in patched 9.X.

There is no support for SACK in 10.X up through 10.20.

Procedure for raising network limits under HPUX 11

HP-UX 11supports PMTU discovery and enables it by default. This is controlled through the ndd setting ip_pmtu_strategy.

Note: Addition (extensive) information is available at ftp://ftp.cup.hp.com/dist/networking/briefs/annotated_ndd.txt

RFC 1323 support is enabled automagically in HP-UX 11. If an application requests a window/socket buffer size greater than 64 KB, window scaling and timestamps will be used automatically.

The default TCP window size in HP-UX 11 remains 32768 bytes and can be altered though ndd and the settings:

    tcp_recv_hiwater_def
    tcp_recv_hiwater_lfp
    tcp_recv_hiwater_lnp
    tcp_xmit_hiwater_def
    tcp_xmit_hiwater_lfp
    tcp_xmit_hiwater_lnp

FTP in HP-UX 11 uses the new sendfile() system call. This allows data to be sent directly from the filesystem buffer cache through the network without intervening data copies.

HP-UX 11 (patches) and 11i (patches or base depending on the revision) have commercial support for SACK (based on feedback from HP - Thanks!)

Here is some ndd -h parm output for a few of the settings mentioned above. For those not mentioned, use ndd -h on an HP-UX 11 system, or consult the online manuals at http://docs.hp.com/

# ndd -h ip_pmtu_strategy

ip_pmtu_strategy:

    Set the Path MTU Discovery strategy: 0 disables Path MTU
    Discovery; 1 enables Strategy 1; 2 enables Strategy 2.

    Because of problems encountered with some firewalls, hosts,
    and low-end routers, IP provides for selection of either
    of two discovery strategies, or for completely disabling the
    algorithm. The tunable parameter ip_pmtu_strategy controls
    the selection.

    Strategy 1: All outbound datagrams have the "Don't Fragment"
    bit set. This should result in notification from any intervening
    gateway that needs to forward a datagram down a path that would
    require additional fragmentation. When the ICMP "Fragmentation
    Needed" message is received, IP updates its MTU for the remote
    host. If the responding gateway implements the recommendations
    for gateways in RFCM- 1191, then the next hop MTU will be included
    in the "Fragmentation Needed" message, and IP will use it.
    If the gateway does not provide next hop information, then IP
    will reduce the MTU to the next lower value taken from a table
    of "popular" media MTUs.

    Strategy 2: When a new routing table entry is created for a
    destination on a locally connected subnet, the "Don't Fragment"
    bit is never turned on. When a new routing table entry for a
    non-local destination is created, the "Don't Fragment" bit is
    not immediately turned on. Instead,

    o  An ICMP "Echo Request" of full MTU size is generated and
       sent out with the "Don't Fragment" bit on.

    o  The datagram that initiated creation of the routing table
       entry is sent out immediately, without the "Don't Fragment"
       bit. Traffic is not held up waiting for a response to the
       "Echo Request".

    o  If no response to the "Echo Request" is received, the
       "Don't Fragment" bit is never turned on for that route;
       IP won't time-out or retry the ping. If an ICMP "Fragmentation
       Needed" message is received in response to the "Echo Request",
       the Path MTU is reduced accordingly, and a new "Echo Request"
       is sent out using the updated Path MTU. This step repeats as
       needed.

    o  If a response to the "Echo Request" is received, the
       "Don't Fragment" bit is turned on for all further packets
       for the destination, and Path MTU discovery proceeds as for
       Strategy 1.

    Assuming that all routers properly implement Path MTU Discovery,
    Strategy 1 is generally better - there is no extra overhead for the
    ICMP "Echo Request" and response. Strategy 2 is available
    only because some routers, or firewalls, or end hosts have been
    observed simply to drop packets that have the DF bit on without
    issuing the "Fragmentation Needed" message. Strategy 2 is more
    conservative in that IP will never fail to communicate when using
    it. [0,2] Default: Strategy 2

# ndd -h tcp_recv_hiwater_def | more

tcp_recv_hiwater_def:

    The maximum size for the receive window. [4096,-]
    Default: 32768 bytes

# ndd -h tcp_xmit_hiwater_def

tcp_xmit_hiwater_def:

    The amount of unsent data that triggers write-side flow control.
    [4096,-] Default: 32768 bytes

http://www.netperf.org/netperf/NetperfPage.html

Procedure for raising network limits on IBM RS/6000 systems under AIX 3.2 or AIX 4.1

See the "no" man page for options; additional information is available in the IBM manual AIX Versions 3.2 and 4.1 Performance Tuning Guide, which is available on AIX machines through the InfoExplorer hypertext interface.

Procedure for raising network limits on IBM MVS systems under the Interlink TCP stack

• TCP SCALE(4) - specifies to support window scaling of 4 bits. Range is 0 (suppress both window scaling and timestamps) to 14 bits.

  If SCALE is not zero, and the user bufferspace is > 65535, negotiating window scaling and timestamps will be attempted.

  If SCALE is not zero, and the remote user negotiates window scaling or timestamps, we will accept those requests.

• FTP IBUF(4 20480) - would specify a receive bufferspace of 81920 bytes, and thus eligible for window scaling and timestamps.

• QUOTE SITE IBUF(num size) - specifies the input bufferspace for file transfers. When the product is larger than 65535, negotiating window scaling and timestamps will be attempted (if SCALE is not zero).

Tuning a Linux 2.4 system

	/proc/sys/net/ipv4/tcp_timestamps
	/proc/sys/net/ipv4/tcp_window_scaling
	/proc/sys/net/ipv4/tcp_sack 

	echo 1 > /proc/sys/net/ipv4/tcp_timestamps
	echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
        echo 1 > /proc/sys/net/ipv4/tcp_sack 

Path MTU discovery can be enabled and disabled using the following boolean sysctl variable:

	/proc/sys/net/ipv4/ip_no_pmtu_disc

http://www.linuxhq.com/kernel/v2.4/doc/networking/ip-sysctl.txt.html
    

Tuning the default and maximum window sizes:

	/proc/sys/net/core/rmem_default   - default receive window
	/proc/sys/net/core/rmem_max       - maximum receive window
	/proc/sys/net/core/wmem_default   - default send window 
	/proc/sys/net/core/wmem_max       - maximum send window

	/proc/sys/net/ipv4/tcp_rmem       - memory reserved for TCP rcv buffers
	/proc/sys/net/ipv4/tcp_wmem       - memory reserved for TCP snd buffers

The following values would be reasonable for path with a large BDP:

       	echo 8388608 > /proc/sys/net/core/wmem_max 
       	echo 8388608 > /proc/sys/net/core/rmem_max 
       	echo "4096 87380 4194304" > /proc/sys/net/ipv4/tcp_rmem
       	echo "4096 65536 4194304" > /proc/sys/net/ipv4/tcp_wmem

You will find a short description in LINUX-SOURCE_DIR/Documentation/networking/ip-sysctl.txt

If you would like these changes to be preserved across reboots, it may be a good idea to add these commands to your /etc/rc.d/rc.local file.

Linux users may want to look into the Web100 project. In addition to providing complete instrumentation of the TCP stack, it has features for doing auto tuning of send and receive buffers.

For paths with a very large BDP, it also has some features that will allow you to get better transfers.

User testimonial: With the tuned TCP stacks it was possible to get a maximum throughput between 1.5 - 1.8 Mbit/s via a 2Mbit/s satellite link, measured with netperf.

Information about tuning for MacOS

I don't have detailed information, however, someone pointed me to a good website with useful information. The URL is http://www.sustworks.com/products/prod_ottuner.html. I don't endorse the product they are selling (since I've never tried it). However, it is available for a free trial, and they appear to do an excellent job of describing perf-tune issues for Macs.

Procedure for raising network limits under NetBSD

      sysctl net.inet.tcp.rfc1323

The maximum socket buffer size can be modified by changing SB_MAX in /usr/src/sys/sys/socketvar.h.

The default socket buffer sizes can be modified by changing TCP_SENDSPACE and TCP_RECVSPACE in /usr/src/sys/netinet/tcp_usrreq.c.

It may also be necessary to increase the number of mbufs, NMBCLUSTERS in /usr/src/sys/arch/*/include/param.h.

Update: It is also possible to set these parameters in the kernel configuration file.

options		SB_MAX=1048576		# maximum socket buffer size
options		TCP_SENDSPACE=65536	# default send socket buffer size
options		TCP_RECVSPACE=65536	# default recv socket buffer size
options		NMBCLUSTERS=1024	# maximum number of mbuf clusters

Procedure for raising network limits under FTP Software (NetManage) OnNet 4.0 for Win95/98

Many default settings, all of the above and more, may be overriden with registry entries. We plan to make available tuning guidelines at "some future time". Also default TCP window may be set with Statistics app which is installed with OnNet Kernel.

The product "readme" discusses changing TCP window size and Initial slow start threshold with the Windows registry.

Statistics also has interesting graphs of TCP/UDP/IP/ICMP traffic. Also IPtrace app is shipped with OnNet Kernel to view unicast / multicast / broadcast traffic (no unicast traffic for other hosts - it does not run in promiscuous mode).

Procedure for raising network limits under SGI systems under IRIX 6.5

Under this version, there are two locations where configuration is done. Although I list the BSD information first, SGI recommends using systune which is described below.

The BSD values are now stored in /var/sysgen/mtune/bsd.

For instance from the file:

* name                  default         minimum   maximum
*
* TCP window sizes/socket space reservation; limited to 1Gbyte by RFC
1323
*
tcp_sendspace                   61440   2048    1073741824
tcp_recvspace                   61440   2048    1073741824

These variables are used similarly to earlier IRIX 5 and 6 versions.

There is now a systune command. This command allows you to configure other networking variables. systune keeps strack of the chances you make in a file called stune so that you can see them all in one place. Also note that changes made using systune are permanent. Here is a sample of things which can be tuned using systune:

/usr/sbin/systune (which is like sysctl for BSD) is what you use for
tuneable values.

 group: net_stp (statically changeable)
        stp_ttl = 60 (0x3c)
        stp_ipsupport = 0 (0x0)
        stp_oldapi = 0 (0x0)

 group: net_udp (dynamically changeable)
        soreceive_alt = 1 (0x1)
        arpreq_alias = 0 (0x0)
        udp_recvgrams = 2 (0x2)
        udp_sendspace = 61440 (0xf000)
        udp_ttl = 60 (0x3c)

 group: net_tcp (dynamically changeable)
        tcp_gofast = 0 (0x0)
        tcp_recvspace = 61440 (0xf000)
        tcp_sendspace = 61440 (0xf000)
        tcprexmtthresh = 3 (0x3)
        tcp_2msl = 60 (0x3c)
        tcp_mtudisc = 1 (0x1)
        tcp_maxpersistidle = 7200 (0x1c20)
        tcp_keepintvl = 75 (0x4b)
        tcp_keepidle = 7200 (0x1c20)
        tcp_ttl = 60 (0x3c)

 group: net_rsvp (statically changeable)
        ps_num_batch_pkts = 0 (0x0)
        ps_rsvp_bandwidth = 50 (0x32)
        ps_enabled = 1 (0x1)

 group: net_mbuf (statically changeable)
        mbretain = 20 (0x14)
        mbmaxpages = 16383 (0x3fff)

 group: net_ip (dynamically changeable)
        tcpiss_md5 = 0 (0x0)
        subnetsarelocal = 1 (0x1)
        allow_brdaddr_srcaddr = 0 (0x0)
        ipdirected_broadcast = 0 (0x0)
        ipsendredirects = 1 (0x1)
        ipforwarding = 1 (0x1)
        ipfilterd_inactive_behavior = 1 (0x1)
        icmp_dropredirects = 0 (0x0)

 group: network (statically changeable)
        netthread_float = 0 (0x0)

 group: inpcb (statically changeable)
        udp_hashtablesz = 2048 (0x800)
        tcp_hashtablesz = 8184 (0x1ff8)

Changes made using systune may or may not require a reboot. This can be easily determined by looking at the 'group' heading for each section of tunables. If the group heading says dynamic, changes can be made on the fly. Group headings labelled static require a reboot.

Finally, the tcp_sendspace and tcp_recvspace can be tuned on a per-interface basis using the rspace and sspace options to ifconfig.

SACK: As of 6.5.7, SACK is included in the IRIX operating system and is on by default.

Procedure for raising network limits under Solaris 2.5

The ndd variable tcp_max_buf specifies the maximum socket buffer size.

To change them, use:

	ndd -set /dev/tcp tcp_max_buf xxx
	ndd -set /dev/tcp tcp_xmit_hiwat xxx
	ndd -set /dev/tcp tcp_recv_hiwat xxx

(Note: I believe xxx should be specified in bytes)

The ndd variable ip_path_mtu_discovery controls the use of path MTU discovery. The default value is 1, which means on.

Note that ndd can also be used to increase the volume of TCP connections available to a machine.

	ndd -set /dev/tcp tcp_conn_req_max 

  tcp_conn_req_max_q 
    default value = 128
    number of connections in ESTABLISHED state
    (3-way handshake completed; not yet accepted)

  tcp_conn_req_max_q0 
    default value =  1024
    number of connections in SYN_RCVD state

Any Registry Values listed appear in:
	HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

Receive Window
	maximum value = 64kB, since window scaling is not supported
	default value = min( max( 4 x MSS, 
				  8kB rounded up to nearest multiple of MSS),
			     64kB) 
	Registry Value: 
		TcpWindowSize

Path MTU Discovery Variables:
		EnablePMTUDiscovery	(default = enabled)
			turn on/off path MTU discovery
		EnablePMTUBHDetect	(default = disabled)
			turn on/off Black Hole detection

Using Path MTU Discovery:
        EnablePMTUDiscovery     REG_DWORD
	Range: 0 (false) or 1 (true)
	Default: 1

    Determines whether TCP uses a fixed, default maximum transmission unit
    (MTU) or attempts to find the actual MTU. If the value of this entry is
    0, TCP uses an MTU of 576 bytes for all connections to computers outside
    of the local subnet. If the value of this entry is 1, TCP attempts to
    discover the MTU (largest packet size) over the path to a remote host.

Using Path MTU Discovery's "Blackhole Detection" algorithm:
        EnablePMTUBHDetect     REG_DWORD
	Range: 0 (false) or 1 (true)
	Default: 0 

    If the value of this entry is 1, TCP tries to detect black hole routers
    while doing Path MTU Discovery. TCP will try to send segments without
    the Don't Fragment bit set if several retransmissions of a segment go
    unacknowledged. If the segment is acknowledged as a result, the MSS will
    be decreased and the Don't Fragment bit will be set in future packets on
    the connection.

    http://rdweb.cns.vt.edu/public/notes/win2k-tcpip.htm