RE: ISCSI: Urgent Flag requirement violates TCP.

[julian_satran@il.ibm.com]

Dante,

All you say makes sense except that the sequence number won't help you the
iSCSI packet header
gets lost. Then you have to either be able to find fast the next one (and
that what the urgent pointer and bit are here for) or pray...

This is all stuff that was repeated several time on this reflector and a
quick search in the Archive helps always.

Julo

"Dante Malagrino'" <dantem@cisco.com> on 14/11/2000 20:32:34

Please respond to "Dante Malagrino'" <dantem@cisco.com>

To:   "Y P Cheng" <ycheng@advansys.com>
cc:   ips@ece.cmu.edu
Subject:  RE: ISCSI: Urgent Flag requirement violates TCP.




At 09:18 AM 11/14/2000 -0800, Y P Cheng wrote:
>Well, the reasons for continuing to move data as fast as the wire speed in
>the face of a sequence hole are: 1) out-of-order reception is considered
>normal and happens often, 2) On a ten gigabit backbone, several
milliseconds
>of delay requires the buffering of several megabytes of data on an
adapter.
>(One megabyte per millisecond).  Incoming data are not limited from a
single
>source.  Many nodes may send and return data to an adapter at the same
time.
>At a gigabyte per second incoming rate, using several megabytes of SRAM
for
>buffering on an adapter is very expensive.

1) what do you mean by "out-of-order reception is considered normal and
happens often"? Last figures I saw talked about 3% of out-of-order TCP
segments on WANs (worst-case). Is it a critical value?

2) with 64MB of RAM, and not supporting Window Scale Option, given a
maximum Window Size of 64k (I have seen it very rarely), we end up having
1k-conns concurrently communicating with a given adapter. With a more
reasonable Window Size (16k), the number of connections is four times more.
Do we expect to see more connections than that? Do we plan to support
Window Scale Option?

>If there is any possibility for a TOE adapter to learn the beginning of an
>iSCSI PDU in face of a sequence hole, while it could try to keep in-order
>delivery of command and status PDUs -- although may not be necessary, the
>data PDUs can be moved quickly to the buffers pre-allocated by application
>software.  Hence, it will greatly reduce the buffering requirement of the
>adapter.

TCP is bytestream oriented, and each packet carries a Sequence Number. Why
can't you save an out-of-order packet in his appropriate location (in the
application buffer), and then put the missing packet in the appropriate
hole? There is no need for copy.

>If a TOE adapter can't move at the speed of the wire, how are we going to
>take advantage of the 10 Gbps media?  On a large WAN with high speed
>backbone connections with 100 millisecond latency, there could be 100
>megabytes of data inflight.  Can we buffer all 100 megabytes on an adapter
>or should we limit the inflight data by set a small TCP window limited by
>the buffer size of the adapter?

Even with large window sizes (64k), how do we have 100MB inflight? See my
previous comment.

>The right design of a TOE adapter is always
>to move data quickly to the buffers already allocated by application
>software and to allow as much data inflight as possible.  To achieve that,
>the TOE adapter needs all the help it can get.  If we can't move data at
the
>wire speed, lets not bother to build 10 Gbps networks.

Isn't the help provided by the TCP sequence number enough?

-- Dante




Dante Malagrino'
Cisco Systems                  Empowering the Internet Generation
170 West Tasman Drive                         Tel. (408) 525 4120
San Jose, CA, 95134-1706                      Fax. (408) 525 4120
                                                  dantem@cisco.com