RE: ISCSI: Urgent Flag requirement violates TCP.

[Black_David@emc.com]

Some form of Matt's longer explanation (see below) about
the effects of coalescence needs to be substituted in
the draft for the sentence:

  "The result is the TCP urgent pointer will point to the first byte of
	the iSCSI message in the TCP segment."

because it's very easy to construct cases in which the
TCP urgent pointer points out of the segment, making the
above sentence false.  In particular, someone constructing
a hardware framer that looks for iSCSI CDBs should not be
mislead into relying on the urgent pointer to identify
a useful (or even worse, the first useful) boundary
in every inbound segment.

Words are also needed to make it absolutely clear that
TCP is not permitted to "deliver" out of order to the
iSCSI layer - as Julian explained, the only point of
this urgent mechanism is to help out framing logic that's
processing the inbound stream, especially if SACK is being
used. Among the disasters that this prevents is iSCSI
completing and sending a response to a command when
the delivery of that command's CDB to the target has
not been ACKed by TCP (e.g., a TCP not using SACK).

Let me expound on this a bit more because it's come up
in the past.  TCP is an ordered delivery mechanism;
data must be "delivered" to the target in order.  In
contrast, SCSI is mostly an unordered mechanism - unless
order is explicitly specified, commands can complete out
of order, and often do on disks and disk arrays.  I
put "delivered" in quotes because the only way to know
that a SCSI command has been "delivered" is to observe
that execution of the command has started (e.g., the
disaster in the previous paragraph where execution completed
on data that was never ACKed by TCP is clear evidence that
someone is not playing by the rules).  So, it's ok
to parse/frame/etc. the data to set it up for execution
of the SCSI command, but DON'T DO ANYTHING until TCP
has "delivered" the command and all the data prior to 
it by sending or being prepared to send the appropriate
ACKs.

Meanwhile, I should note that a TCP stack may respond to having
urgent data presented to it by sending immediately (i.e., and
not waiting for the rest of the CDB).  For alignment reasons,
using 1 byte of urgent data may not be the best choice.

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------


> -----Original Message-----
> From:	Matt Wakeley [SMTP:matt_wakeley@agilent.com]
> Sent:	Tuesday, November 07, 2000 4:44 PM
> To:	Douglas Otis
> Cc:	julian_satran@il.ibm.com; ips@ece.cmu.edu
> Subject:	Re: ISCSI: Urgent Flag requirement violates TCP.
> 
> Douglas Otis wrote:
> 
> > ISCSI: Urgent Flag requirement violates TCP.
> 
> No it doesn't.
> 
> > Julian,
> >
> > The result indicated within the iSCSI draft on page 14 will require TCP
> to
> > be modified for this statement to be true.  By requiring every message
> to be
> > marked, a continued transparent update of the pointer will obscure the
> > urgent pointer by continued coalescence.  This obscuring coalescence
> will
> > occur at both the sender and receiver.  Only a state between normal and
> > urgent is signaled at the receiver and not the actual pointer.  Unless
> there
> > is an intent to modify TCP, this scheme offers no benefit even if
> offending
> > portions of the urgent proposal is removed.
> 
> We don't care if sometimes coalescing happens.  The point is, that
> somewhere
> down the stream, an urgent pointer will point to the beginning of a new
> iSCSI
> message and messages after this point can be processed.  All messages
> between
> the point of the dropped segment and this point will of course have to be
> retransmitted.
> 
> 
> >    The iSCSI protocol uses the urgent bit in the TCP header to delineate
> >    iSCSI messages. The first byte of every iSCSI message MUST be marked
> >    "urgent".  The result is the TCP urgent pointer will point to the
> >    first byte of the iSCSI message in the TCP segment.
> >
> > This pointer will be skewed by the size of the send buffer (beyond the
> TCP
> > segment) and prevent normal use unless again TCP is modified to comply
> with
> > stated expectations.  I would advise complete removal of this Urgent
> flag
> > requirement if to remain within the WG charter as only a modified TCP
> could
> > use this feature in the intended manner.  This intended manner is
> plainly
> > spelled out with concerns about BSD vs. RFC 1122 compliance. There is no
> > benefit from an occasional toggle between normal and urgent mode.  If
> you
> > wish to use a datagram or record based protocol, you are advised to
> review
> > RFC 2960.
> 
> The intended manner that is to be implemented is this:
> 
> 1. iSCSI sends the first byte of the iSCSI message to TCP, with the byte
> marked
> "urgent".  You can do this today using any off the shelf TCP stack,
> including MS
> windows.
> 2. iSCSI sends the rest of the bytes of the iSCSI message to TCP with
> normal
> delivery.
> 
> If TCP coalesces, fine.  But eventually, a TCP segment will be sent that
> will
> contain a byte referenced by the urgent pointer.  At this point, the
> remote side
> (and any LAN analyzers on the network) will be able to "sync up" on iSCSI
> messages.
> 
> Remember, this "urgent pointer" stuff is *only* to be used in the cases
> where a
> TCP segment is dropped, not for normal delivery.
> 
> So, there is *no* violation of any RFC or WG charter.
> 
> >
> >
> > Doug
> 
> -Matt Wakeley
> Agilent Technologies