Poor throughput from London to New York GN2 MP

PTS Case 19

Background

TCP throughput below expected value has been experienced between GN2 Measurement Point in New York (US) and the MP in London (UK). Some initial tests have been made and they show that TCP performs much better on UK-NY path than in opposite direction (~500 Mbits/s vs. 20Mbits/s). The poor performance is present either for Premium IP traffic and LBE traffic.

Investigation

The investigation concentrated on why PIP traffic was being affected by LBE (the converse is of course expected and indeed desired). By chance this problem came to light just as a new trans-Atlantic circuit was being deployed, and it was possible to carry out some detailed testing on an otherwise unloaded STM16 between London and NYC (for the short period before the provider terminated it). The specific cause of the original very low TCP transfer rates was not confirmed (but may have been a buggy kernel, see below), but the cause of the PIP/LBE interference was determined to be caused by packet re-ordering - when there was a large LBE flow in progress then a new PIP flow would get re-ordered to the extent that the sender inferred packet loss and restricted its transmit rate. By changing the number of packets that could be re-ordered before packet loss was assumed, new PIP flows could grow as required and supress the incumbent LBE flow.

Outcome

It was determined that the appropriate configuration for the MPs was to increase the TCP re-ordering parameter from the default 3 to some larger value (say 20) and to use the BIC algorithm, not the H-TCP algorithm (which in Linux kernel 2.6.13.2b is buggy). This having been done, the London MP was able to achieve a 800 Mbps plus in each direction.

– Main.TobyRodwell - 31 Jan 2007