RE: ISCSI: Urgent Flag requirement violates TCP.

["Y P Cheng" <ycheng@advansys.com>]

> From: Vern Paxson [mailto:vern@ee.lbl.gov]
> Sent: Monday, November 13, 2000 10:22 PM
>
> Why is it so important to go as fast as possible in the face of a sequence
> hole?  If the hole is due to packet loss, then the sender is going to have
> to cut their sending rate in half next RTT anyway.  If it's due to a
> reordering, then measurement studies have found the reordering generally
> resolves in a few msec.  And in this case will you turn off duplicate
> acks, which might otherwise trigger a congestion response anyway?
>
> More generally, along what sort of possibly lossy, possibly reordering
> paths will iSCSI be trying to squeeze out Gbps+ rates?

Well, the reasons for continuing to move data as fast as the wire speed in
the face of a sequence hole are: 1) out-of-order reception is considered
normal and happens often, 2) On a ten gigabit backbone, several milliseconds
of delay requires the buffering of several megabytes of data on an adapter.
(One megabyte per millisecond).  Incoming data are not limited from a single
source.  Many nodes may send and return data to an adapter at the same time.
At a gigabyte per second incoming rate, using several megabytes of SRAM for
buffering on an adapter is very expensive.

If there is any possibility for a TOE adapter to learn the beginning of an
iSCSI PDU in face of a sequence hole, while it could try to keep in-order
delivery of command and status PDUs -- although may not be necessary, the
data PDUs can be moved quickly to the buffers pre-allocated by application
software.  Hence, it will greatly reduce the buffering requirement of the
adapter.

If a TOE adapter can't move at the speed of the wire, how are we going to
take advantage of the 10 Gbps media?  On a large WAN with high speed
backbone connections with 100 millisecond latency, there could be 100
megabytes of data inflight.  Can we buffer all 100 megabytes on an adapter
or should we limit the inflight data by set a small TCP window limited by
the buffer size of the adapter?  The right design of a TOE adapter is always
to move data quickly to the buffers already allocated by application
software and to allow as much data inflight as possible.  To achieve that,
the TOE adapter needs all the help it can get.  If we can't move data at the
wire speed, lets not bother to build 10 Gbps networks.