Re: TCP limitations (was Re: ISCSI: Urgent Flag requirement violates TCP.)

[julian_satran@il.ibm.com]

Vern,

Thanks for your thoughtful insight and references.
It took me some time to get through them but I think that I understand your
line of thought.
It is true that the TCP stack has no way of distinguishing between an error
and a congestion signal
and will halve the congestion window at one of the "bad" signals like
Triple-Ack or Timeout whether it is caused by a link error or congestion or
reordering.
 However the model is developed for "congestion bound" networks and its
value for other contexts has to be examined on a case for case basis.

This explains why the formula (or its more refined version) does not
contain the link bandwidth (assuming that links are congestion bound).
Assuming that we have a very fast network with a very low error rate we can
achieve a state in witch the congestion window is not limiting the
transmission and errors while reducing the congestion window will have no
effect on the transmission rate.

As a simple example consider a set of links of 1 GBs connected to a
backbone of 10Gbs.
Congestion control (by RED or tail drop) will kick in only when the
aggregate rate is close to or over 10Gbps
while an error will have almost no effect when the aggregate rate is say
2Gbps.

Even in case in which the network is error bound your assumption that we
can bound the receive buffer as a function of the expected throughput
violate the assumptions of the model you quote (assuming the model holds -
and I am not sure it does as some averages are calculated based on
congestion behavior) that the receiver has an infinite window and will
lower dramatically the throughput - as the "apparent" error rate is
increased by this effect.

The numbers you quote for p are also on the "pessimistic" range as BERs of
10**-12 are quoted for optical networks and that gives a packet error rate
of 10**-8 .

But you pointed us to a true issue - error can have a "super-linear" effect
on throughput as TCP interprets every error as a congestion signal and the
fact that error rates have not decreased even linearly with link rate
increases could spell trouble for TCP on high-speed networks.

Regards,
Julo


Vern Paxson <vern@ee.lbl.gov> on 21/11/2000 09:42:55

Please respond to Vern Paxson <vern@ee.lbl.gov>

To:   "Douglas Otis" <dotis@sanlight.net>
cc:   "Bernard Aboba" <aboba@internaut.com>, "Y P Cheng"
      <ycheng@advansys.com>, Black_David@emc.com, ips@ece.cmu.edu
Subject:  TCP limitations (was Re: ISCSI: Urgent Flag requirement violates
      TCP.)




> Your assumptions about reordering is not accurate for WAN.  Should there
be
> more than one long-haul fiber in use, common in metro areas, the
disparate
> physical routes may induce out of sequence events.  Is such a case, SACK
> reduces the negative effect.

SACK as currently deployed does not actually help all that much with
reordering.  The receiver will still generate duplicate acks and the
sender will misinterpret them as indicating packet loss, leading to
a congestion response of halving the sending rate.  DSACK (RFC 2883),
once deployed, will give the sender a means to recognize when it made
a mistake, but work on developing the algorithm the sender uses to do so,
and in particular how it should recover from the mistake, is not yet
in the IETF pipeline.

More generally, I think the ISCSI design assumptions about needing to go
at gigabit rates over a WAN, and hence the need for using Urgent to do
framing when there's a sequence hole, miss a basic TCP reality.  TCP's
congestion control gives an explicit relationship between BW, the sustained
throughput you can attain, and p, the packet loss rate:

                       MSS   1
     BW = sqrt(3/2) --- -------
                       RTT sqrt(p)

where RTT is the round trip time and MSS is the packet size.

If we can control things so that p=10^-4, i.e., only one packet in 10,000
is lost (either due to congestion, or corruption), then for MSS=1500 bytes
and RTT=100 msec, we get:

     BW = 1.837 MB/s.

That's it - under 2 MB/s.

So if you want to go fast, you have to have *much less* than one sequence
hole per 10^4 packets.  But the Urgent pointer only helps when you happen
to have a sequence hole.  So how in practice can it be worth the effort?

          Vern