ALTQ/RED Performance

How RED works with TCP with or without ECN.

Test Date: 98/01/30

Test System Configuration

The following measurements were done with three PentiumPro machines (all 200MHz with 440FX chipset) running FreeBSD-2.2.5/altq-0.4.3.

Host A is a source, host B is a router, and host C is a sink. RED is enabled only on the interface of host B connected to host C. The link used is 155Mbps ATM (134Mbps over AAL5), but the link speed between host A and host B is set to 45Mbps, and the link speed between host B and host C is set to 6Mbps by the hardware shaper to create a bottleneck.

The tests use the TCP test mode of Netperf benchmark program. Four 20-second-long TCP flows are sent from host A to host C; the flows start 3-second-apart from each other. (host B is also a source in trace-5 to trace-8).

Figure 1 Test System Configuration
The following 8 conditions are measured.

Trace-1: No RED
Trace-2: simple RED (Drop Only)
Trace-3: RED with ECN (Packet Marking Only)
Trace-4: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)
Trace-5: 2 Sources: No RED
Trace-6: 2 Sources: simple RED (Drop Only)
Trace-7: 2 Sources: RED with ECN (Packet Marking Only)
Trace-8: 2 Sources: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)

RED statistics

RED statistics are taken at host B.

Table 1. RED statistics
Trace No.	xmit pkts	drop pkts (forced, unforced)	marked pkts
trace-1: No RED	2853	49 (-, -)	-
trace-2: Drop Only	2833	21 (0, 21)	0
trace-3: ECN Only	2844	0 (0, 0)	24
trace-4: Mixed	2849	11 (0, 11)	11
trace-5: 2 SRC (No RED)	3084	54 (-, -)	-
trace-6: 2 SRC (Drop)	2854	15 (0, 15)	0
trace-7: 2 SRC (ECN)	2859	0 (0, 0)	29
trace-8: 2 SRC (Mixed)	2879	9 (0, 9)	12

Traffic Traces

The traffic traces are taken at host C using Tele Traffic Tapper (that uses BPF). The granularity of the traces is 400 ms. 4 TCP flows use src-port 6790 - 6793. The uppermost red line shows the total traffic.

The raw tcpdump traces for those tests are also available.

Trace-1: No RED

When there is no active queue management, packet drop occurs without any control. Although the link is kept full, traffic behavior is unpredictable in this situation. When a flow loses packets, it also loses its share of the link bandwidth.

Figure 2 Trace-1: No RED

Trace-2: simple RED (Drop Only)

When RED is used, the bandwidth allocation becomes fair. The flows sometimes lose packets but, in many cases, the flows keep their share by the fast recovery mechanism. However, flow-6790 experienced a retransmit timeout at time 29. The fast recovery mechanism fails when the sender's window is not large enough, or multiple packets get dropped in a window (in a Reno derived TCP).

Figure 3 Trace-2: Drop Only

Trace-3: RED with ECN (Packet Marking Only)

When ECN is used, link-sharing becomes more stable.

Figure 4 Trace-3: ECN Only

Trace-4: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)

This test illustrates what will happen when ECN is introduced to the existing network. 2 flows (6791 and 6793) are ECN-capable, but the other 2 (6790 and 6792) are not ECN-capable. RED drops non-ECN-capable packets instead of marking. Both of non-ECN-capable flows (6790 and 6792) experienced retransmit timeouts at time 28, 30 and 35.

Figure 5 Trace-4: Mixed case

Comments by Sally Floyd: My guess would be that under medium congestion, ECN-capable TCP might get higher bandwidth then non-ECN-capable TCP, because it is less likely to have to wait for a retransmit timeout. However, my conjecture would be that the differences in throughput are neglibable in mild congestion. The non-ECN-capable TCP has a packet **dropped** and the ECN-capable TCP has a packet **marked**, but both TCP's reduce their congestion window by half.

Effects of different RTT

I couldn't produce large RTT difference, but tried to use host B (router) as another source of flows. RTT between host A and host C is 0.750 ms and RTT between host B and host C is 0.539 ms (measured by ping). So, the difference is small. (note that when the link is full, RTT can be more than 100ms, partly because the queues are full and partly because the process is not scheduled.)

I tested the same settings used in trace-1 to trace-4 but 2 flows (6791 and 6793) are sent from host B. The other 2 flows (6790 and 6792) were sent from host A.

Trace-5: 2 Sources: No RED

The flows oscillate more than trace-1.

Figure 6 Trace-5: No RED from 2 Sources

Trace-6: 2 Sources: simple RED (Drop Only)

More retransmit timeouts are observed than trace-2.

Figure 7 Trace-6: Drop Only from 2 Sources

Trace-7: 2 Sources: RED with ECN (Packet Marking Only)

ECN works much better than dropping (trace-6).

Figure 8 Trace-7: ECN Only from 2 Sources

Trace-8: 2 Sources: RED with ECN, 2 ECN-capable TCP, 2 non-ECN TCP (Mixed)

One of non-ECN-capable flows (6792) experienced a retransmit timeout.

Figure 9 Trace-8: Mixed case from 2 Sources

Summary

Although these tests are limited in many ways, it is clear that RED works pretty well with TCP. Even simple RED (without ECN) can provide fairness to some extent and can provide triggers for fast recovery. ECN makes these features more stable, and will reduce end-to-end delay by preventing TCP from retransmission timeouts.