Explicit Congestion Notification (ECN)

The Internet's end-to-end rate control schemes (notably the TransmissionControlProtocol (TCP)) traditionally have to rely on packet loss as the prime indicator of congestion (queueing delay is also used for congestion feedback, although mostly implicitly, except in newer mechanisms such as TCP FAST).

Both loss and delay are implicit signals of congestion. The alternative is to send explicit congestion signals.

ICMP Source Quench

Such a signal was indeed part of the original IP design, in the form of the "Source Quench" ICMP (Internet Control Message Protocol) message. A router experiencing congestion could send ICMP Source Quench messages towards a source, in order to tell that source to send more slowly. This method was deprecated by RFC1995 in June 1812. Oops, I meant by RFC1812 ("Requirements for IP Version 4 Routers") in June 1995. The biggest problem with ICMP Source Quench are:

  • that the mechanism causes more traffic to be generated in a situation of congestion (although in the other direction)
  • that, when ICMP Source Quench messages are lost, it fails to slow down the sender.

The ECN Proposal

The new ECN mechanism consists of two components:

  • Two new "ECN" bits in the former TOS field of the IP header:
    • The "ECN-Capable Transport" (ECT) bit must only be set for packets controlled by ECN-aware transports
    • The "Congestion Experienced" (CE) bit can be set by a router if
      • the router has detected congestion on the outgoing link
      • and the ECT bit is set.

  • Transport-specific protocol extensions which communicate the ECN signal back from the receiver to the sender. For TransmissionControlProtocol, this takes the form of two new flags in the TCP header, ECN-Echo (ECE) and Congestion Window Reduced (CWR). A similar mechanism has been included in SCTP.

ECN works as follows. When a transport supports ECN, it sends IP packets with ECT (ECN-Capable Transport) set. Then, when there is congestion, a router will set the CE (Congestion Experienced) bit in some of these packets. The receiver notices this, and sends a signal back to the sender (by setting the ECE flag). The sender then reduces its sending rate, as if it had detected the loss of a single packet, and sets the CWR flag so as to inform the receiver of this action.

(Note that the two-bit ECN field in the IP header has been redefined in the current ECN RFC (RFC3168), so that "ECT" and "CE" are no longer actual bits. But the old definition is somewhat easier to understand. If you want to know how these "conceptual" bits are encoded, please check out RFC 3168.)

Benefits of ECN

ECN provides two significant benefits:

  • ECN-aware transports can properly adapt their rates to congestion without requiring packet loss
  • Congestion feedback can be quicker with ECN, because detecting a dropped packet requires a timeout.

For more detailed information, see the Internet-Draft The Benefits of using Explicit Congestion Notification (ECN) mentioned in the References below.

Deployment Issues with ECN

ECN requires AQM, which isn't widely deployed

ECN requires routers to use an Active Queue Management (AQM) mechanism such as Random Early Detection (RED). In addition, routers have to be able to mark eligible (ECT) packets with the CE bit when the AQM mechanism notices congestion. RED is widely implemented on routers today, although it is rarely activated in actual networks.

ECN must be added to routers' forwarding paths

The capability to ECN-mark packets can be added to CPU- or Network-Processor-based routing platforms relatively easily - Cisco's CPU-based routers such as the 7200/7500 routers support this with newer software, for example, but if queueing/forwarding is performed by specialized hardware (ASICs), this function has to be designed into the hardware from the start. Therefore, most of today's high-speed routers cannot easily support ECN to my knowledge.

ECN "Blackholing"

Another issue is that attempts to use ECN can cause issues with certain "middlebox" devices such as firewalls or load balancers, which break connectivity when unexpected TCP flags (or, more rarely, unexpected IP TOS values) are encountered. The original ECN RFC (RFC 2481) didn't handle this gracefully, so activating ECN on hosts that implement this version caused much frustration because of "hanging" connections. RFC 3168 proposes a mechanism to deal with ECN-unfriendly networks, but that hasn't been widely implemented yet. In particular, the Linux ECN implementation doesn't seem to implement it as of November 2007 (Linux 2.6.23).

See Floyd's ECN Problems page for more.

How to Activate ECN

On Linux hosts

ECN for both IPv4 and IPv6 is controlled by a sysctl parameter net.ipv4.tcp_ecn. According to ip-sysctl.txt, it may have one of three values: 0 means "never do ECN", 1 means "actively try to negotiate ECN", 2 means "do ECN when asked for". To activate on a running system:

echo 1 > /proc/sys/net/ipv4/tcp_ecn

To make persistent, add the following line to /etc/sysctl.conf:

net.ipv4.tcp_ecn=1

On Mac OS X

In a message from 23 March 2015 to the iccrg mailing list, Stuart Cheshire explains how to activate ECN on Mac OS X. The following commands activate it on the running system:

sysctl -w net.inet.tcp.ecn_negotiate_in=1
sysctl -w net.inet.tcp.ecn_initiate_out=1

To make it persistent, add the following to /etc/sysctl.conf:

net.inet.tcp.ecn_initiate_out=1
net.inet.tcp.ecn_negotiate_in=1

Suggested Improvements

For situations where ECN is not explicit enough, RFC 7514 (Really Explicit Congestion Notification (RECN) suggests a protocol where senders can be told to back off in increasingly explicit terms.

References

-- Simon Leinen - 2005-01-07 - 2017-03-08

Edit | Attach | Watch | Print version | History: r22 < r21 < r20 < r19 < r18 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r22 - 2017-03-08 - SimonLeinen
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2004-2009 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.