RE: ISCSI: Urgent Flag requirement violates TCP.

["Douglas Otis" <dotis@sanlight.net>]

Y.P.

> > From: Douglas Otis [mailto:dotis@sanlight.net]
> > Sent: Monday, November 13, 2000 10:24 AM
> >
> > Y.P.
> >
> > TCP does not technically allow zero copy as PDU headers may be
> > split across segments and urgent pointers may coalesce beyond
> > received segments.  You are advocating modification to TCP
> > specifically for iSCSI and appear to advocate segment alignment
> > for PDU headers with a one packet at a time standard.
> > Do you wish to see the TCP sender and receiver modified as
> > indicated within the proposal?
>
> The TOE is designed to deal the segment alignment and coalescing of
> segments.  All I said was that the urgent pointer helps the parsing of the
> segment not changing the alignment.  Without the urgent pointer we need to
> wait for all missing segments before parsing.  Missing data
> segments do not
> affect the parsing at all.

Without receiving a complete status count of all PDU types, there would be
little to indicate what was contained within a missing segment.  There is
not a "data segment" unless you are suggesting segment alignment which you
say is not the case, but your rebuttal appears otherwise.

> We know the missing segments are data, the
> buffering requirements for the TOE is dramatically reduced.  For 10 Gb
> Ethernet, there are gigantic TCP windows when latency time on transmission
> is in hundreds of milliseconds.  (All data segments are passed through to
> the application software directly.  Only segments containing
> iSCSI messages
> are important to the TOE adapter.)  No, I don't wish the TCP sender or
> receiver to be modified as I have said that the TOE adapter would
> work with
> all clients and servers implemented in traditional TCP.  Only the
> implementation of the TCP software where the TOE adapter is
> installed needs
> to be modified for adopting the zero-copy function.  Since there is no TCP
> changes, this WG proposal requires no descriptions beyond the
> iSCSI itself.

Should the adapter not be able to buffer segments received following a lost
segment, SACK should not be used.  As any lost segment will have a
significant impact on throughput, picking-up from the missing segment allows
for minimal buffering while remaining compliant.  As this atypical record
marking option is a burden to vanilla implementations, complete this
proposal by documenting this TOE-iSCSI adapter interface.  Clearly, there is
no justification for these additional requirements without this modified
receiver.  Documentation allows the adapter industry to make interchangeable
devices at the API level.  "In the Box" is not justification for no
documentation of this rather complex TOE function you wish supported.  At
least the process of documenting this function may indicate where things can
be improved or where things go wrong.  As the queue mode is indicated within
each command, no command can be executed without all prior commands seen.

As only a single immediate write can be made in flight with iSCSI, long fat
pipes are going to be slowed anyway.  With multiple connections implied by
the iSCSI proposal, this Zero Copy with missing segments concern is also
significantly reduced as there would never be a fat anything.

> > In your view TCP software must be modified so why stop at
> > specifying the urgent pointer, include frame alignment.
> Clearly, you view
> > wire compatibility the only constraint.  Is TCP just a wire
> specification?
>
> There are many TCP implementations.  As Matt has said many times, these
> implementations are "inside the box".  TCP does not specify
> "inside the box"
> implementations.

TCP does define "inside the box" at the API level.  Without this level of
documentation, there would be little uniformity in network support.  Simply
document the changes you wish to see supported.  TCP is not a wire
specification nor is SCSI.  Each protocol does not end at the end of the
wire.  It is not a free-for-all inside the box as you imply.  Once you
specify an API, you are free to adhere to this standard or not.  At least
there is a baseline for implementation and expectations of how this new
protocol works.

Doug


> Our discussions herein is limited to "inside the box."
> Yes, the adapter does do out-of-order and missing frame detection, frame
> alignment, flow control and congestion avoidance for TCP.  However, in
> dealing with the data buffers pre-allocated by application software, the
> flow control is greatly simplified.  Congestion avoidance and timeouts for
> missing segments are totally different stories.  All TCP implementations,
> including the TOE adapter, must follow a number of different
> RFC's that were
> kindly pointed out to me by many folks in this WG.
>
> Y.P. Cheng, CTO, ConnectCom Solutions Corp.