TCP limitations (was Re: ISCSI: Urgent Flag requirement violates TCP.)

[Vern Paxson <vern@ee.lbl.gov>]

> Your assumptions about reordering is not accurate for WAN.  Should there be
> more than one long-haul fiber in use, common in metro areas, the disparate
> physical routes may induce out of sequence events.  Is such a case, SACK
> reduces the negative effect.

SACK as currently deployed does not actually help all that much with
reordering.  The receiver will still generate duplicate acks and the
sender will misinterpret them as indicating packet loss, leading to
a congestion response of halving the sending rate.  DSACK (RFC 2883),
once deployed, will give the sender a means to recognize when it made
a mistake, but work on developing the algorithm the sender uses to do so,
and in particular how it should recover from the mistake, is not yet
in the IETF pipeline.

More generally, I think the ISCSI design assumptions about needing to go
at gigabit rates over a WAN, and hence the need for using Urgent to do
framing when there's a sequence hole, miss a basic TCP reality.  TCP's
congestion control gives an explicit relationship between BW, the sustained
throughput you can attain, and p, the packet loss rate:

                       MSS   1
	BW = sqrt(3/2) --- -------
                       RTT sqrt(p)

where RTT is the round trip time and MSS is the packet size.

If we can control things so that p=10^-4, i.e., only one packet in 10,000
is lost (either due to congestion, or corruption), then for MSS=1500 bytes
and RTT=100 msec, we get:

	BW = 1.837 MB/s.

That's it - under 2 MB/s.

So if you want to go fast, you have to have *much less* than one sequence
hole per 10^4 packets.  But the Urgent pointer only helps when you happen
to have a sequence hole.  So how in practice can it be worth the effort?

		Vern