This site has been archived. For information on the GN Project’s eduPERT initiative please visit https://archive.geant.org/projects/gn3/geant/services/edupert/Pages/Home.html

Linux Traffic Shaping Example

The following is a sample configuration from a real network. The host in question is connected via 100 Mb/s Fast Ethernet to a "branch office" LAN (university institute). The "branch office" is connected to the main campus LAN via a 40 Mb/s CBR (Constant Bit Rate) ATM (Asynchronous Transfer Mode) PVC (Permanent Virtual Channel/Connection/Circuit). The host acts as an NFS-over-TCP client, both with servers on the local (branch office) LAN segment and with servers on the main campus.

The Issue

It was observed that write performance from the host to NFS servers on the main campus was limited to about 3 Mb/s, far below the bottleneck capacity of 40 Mb/s.

The issue was diagnosed as being related to bursty loss. Apparently when the client started sending larger bursts of traffic over the ATM circuit to the main campus, the ATM network dropped excess traffic in a way that caused bursty loss, which in turn caused TCP to severely back off its sending rate.

Solution

After some experimentation, the following QoS policy was configured on the host. On-net traffic (matching the U32 filter) is limited to 100 Mb/s, which means that it is only limited by the interface speed. Other traffic - which is sent over the 40 Mb/s ATM circuit - is limited to 43 Mb/s. Note that TSO is turned off using ethtool , because traffic shaping doesn't coexist well with it. Apparently, traffic shaping won't prevent the bursts resulting from TSO's bundling of packets.

With this configuration, NFS write rate comes close to 40 Mb/s, provided that the nosync option is used for the NFS mounts.

IF=lan
SPEED=43

ethtool -K $IF tso off

tc qdisc del dev $IF root

tc qdisc add dev $IF root handle 1: htb default 11

tc class add dev $IF parent 1: classid 1:10 htb rate 100mbit
tc class add dev $IF parent 1: classid 1:11 htb rate ${SPEED}mbit

U32="tc filter add dev $IF parent 1: protocol ip prio 1 u32"
$U32 match ip dst 172.16.224.0/25 flowid 1:10

tc  qdisc add dev $IF parent 1:10 handle 10: sfq quantum 1400b perturb 2
tc  qdisc add dev $IF parent 1:11 handle 11: sfq quantum 1400b perturb 2

– Main.SimonLeinen - 21 Jul 2006 (based on input from Tobi Oetiker)

  • No labels