Reduce TIME_WAIT Socket Connections

Sometime, you’ll run across an Apache server that always has tons of TIME_WAIT connections just seeming to hang out. While these don’t take up as many resources as an ESTABLISHED connection, why keep them around so long? This will show you how to identify how many you have, and how to tell your server to reduce them, reuse and recycle them

  1. Identify how many TIME_WAITs you have hanging out

    netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n
    
  2. Get current values

    cat /proc/sys/net/ipv4/tcp_fin_timeout
    cat /proc/sys/net/ipv4/tcp_tw_recycle
    cat /proc/sys/net/ipv4/tcp_tw_reuse
    
  3. Change values

    echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
    echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
    echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
    
    # with linux 2.6 or higher
    sysctl -w net.ipv4.tcp_fin_timeout=30
    sysctl -w net.ipv4.tcp_tw_recycle=1
    sysctl -w net.ipv4.tcp_tw_reuse=1
    
  4. Read more Linux TCP parameters

    • TCP_FIN_TIMEOUT

      This setting determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. During this TIME_WAIT state, reopening the connection to the client costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, making more resources available for new connections. Adjust this in the presence of many connections sitting in the TIME_WAIT state (default: 60 seconds, recommended 15-30 seconds):

      # sysctl.conf syntax:
      # net.ipv4.tcp_fin_timeout = 15
      echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
      
    • TCP_KEEPALIVE_INTERVAL

      This determines the wait time between isAlive interval probes (default: 75 seconds, recommended: 15-30 seconds):

      # sysctl.conf syntax:
      # net.ipv4.tcp_keepalive_intvl = 30
      echo 30 > /proc/sys/net/ipv4/tcp_keepalive_intvl
      
    • TCP_KEEPALIVE_PROBES

      This determines the number of probes before timing out (default: 9, recommended 5):

      # sysctl.conf syntax:
      # net.ipv4.tcp_keepalive_probes = 5
      echo 5 > /proc/sys/net/ipv4/tcp_keepalive_probes
      
    • TCP_KEEPALIVE_PROBES

      It enables fast recycling of TIME_WAIT sockets. The default value is 0 (disabled). The sysctl documentation incorrectly states the default as enabled. It can be changed to 1 (enabled) in many cases. Known to cause some issues with hoststated (load balancing and fail over) if enabled, should be used with caution (boolean, default: 0):

      # sysctl.conf syntax:
      # net.ipv4.tcp_tw_recycle = 1
      echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
      
    • TCP_TW_REUSE

      This allows reusing sockets in TIME_WAIT state for new connections when it is safe from protocol viewpoint. Default value is 0 (disabled). It is generally a safer alternative to tcp_tw_recycle (boolean, default: 0):

      # sysctl.conf syntax:
      # net.ipv4.tcp_tw_reuse = 1
      echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
      

      Note: The tcp_tw_reuse setting is particularly useful in environments where numerous short connections are open and left in TIME_WAIT state, such as web servers. Reusing the sockets can be very effective in reducing server load.

Comments

Popular posts from this blog