Re: iSCSI : digest error handling violates EMDP/InDataOrder

[julian_satran@il.ibm.com]

2.7.5 will read:

   The Buffer Offset field contains the offset of this PDU payload data
   against the complete data transfer. The sum of the buffer offset and
   length should not exceed the expected transfer length for the command.

   Data ordering is governed by a disconnect-reconnect mode page bit
   (EMDP). If this bit is 0 the data packets must be delivered in
   increasing buffer offset order and data overlays are forbidden.

   3.1.2 will read:

   Data PDUs can be in any order (EMDP = 1) or at continuously increasing
   addresses (EMDP = 0).
   EMDP can also be set by a text-mode key=value pair (DataOrder).

   The appendix will have:

   Use: LO
   Who: Initiator and Target

   DataOrder=<yes|no>

   The default is yes but targets MAY support no.

   No is used by iSCSI to indicate that the data PDUs can be in any order
   (EMDP = 1). Yes is used to indicate that data PDUs have to be at
   continuously increasing addresses (EMDP = 0).

   R2T will contain:

   A R2T MAY be answered with one or more SCSI Data-out PDUs with a
   matching Target Transfer Tag. If a R2T is answered with a single Data
   PDU, the Buffer Offset in the Data PDU MUST be the same as the one
   specified by the R2T. The data length of the Data PDU MUST not exceed
   the Desired Data Length specified in R2T. If the R2T is answered with a
   sequence of Data PDUs the Buffer Offset and Length must be within the
   range of those specified by R2T, the last PDU should have the F bit set
   to 1, the Buffer Offsets and Lengths for consecutive PDUs should form a
   continuous non-overlapping range and the PDUs should be sent in
   increasing offset order.

   The target may send several R2T PDUs (up to a negotiated number) and
   thus have a number of data transfers pending.  Within a connection,
   outstanding R2Ts MUST be fulfilled by the initiator in the order in
   which they where received.

   Buffer offset ordering in consecutive R2Ts is governed by EMDP.  If EMDP
   is 0 consecutive R2Ts SHOULD refer to continuous non-overlapping ranges.
   However, a target MAY send out-of-order R2Ts (e.g., for recovery) and an
   initiator MAY choose to terminate a command when receiving an
   out-of-order R2T that in can't fulfill, with an appropriate response
   (indicating service delivery subsystem failure) and aborting the command
   at the target.

   Julo

Santosh Rao <santoshr@cup.hp.com> on 04-05-2001 03:35:03

Please respond to Santosh Rao <santoshr@cup.hp.com>

To:   Julian Satran/Haifa/IBM@IBMIL
cc:   ips@ece.cmu.edu
Subject:  Re: iSCSI : digest error handling violates EMDP/InDataOrder




In previous exchanges in this thread, it was stated that EMDP only
governs data order within a sequence. This, to me [and others from FC
world, as was noted by Bob] reads as the definition of Random relative
Offset [as defined by bob].

However, Section 2.7.5 on Buffer Offset states that :
"Output data within a burst MUST be delivered in increasing buffer
offset order".

This contradicts the above by indicating that iSCSI forbids Random
Relative Offset.

So, does EMDP apply to WRITEs or does it not (?).

Also, does iSCSI allow data overlay or is this prohibited ? I'd suggest
the following be clearly described in the draft :

a) iSCSI defines EMDP to govern the order of data PDUs within a SCSI
command for both READ and WRITE operations. An EMDP setting of 0 implies
data overlay is forbidden.

b) The out-of-order transmission of a given sequence of WRITE data PDUs
in response to an R2T (the random relative offset property) be
explicitly dis-allowed by iSCSI [which is currently stated in Section
2.7.5].

Regards,
Santosh


julian_satran@il.ibm.com wrote:
>
> Santosh,
>
> Let's take a systematic approach to it.
>
> Restriction on data ordering are required if the source or the
destination
> of the data is unable to deliver or take data data in any order other
that
> sequential.
>
> Semiconductor or other direct access memories don't  have this
restriction.
>
> Tapes and other sequential media do have this type of restriction and so
> some streaming devices.
>
> If the restricted device is a target of a SCSI operation with an
> unrestricted initiator then:
>
> a. on reads the target can always ship its data in sequential order
> b. on writes the target can  always request the data in sequential order
>
> However if the restricted device is an initiator then:
>
> a. on reads the initiator will request the target to send the data in
order
> b. on writes the restricted initiator will have to get the R2Ts in order
> from the target and will be able to support data recovery through an R2T
> only if it has enough buffered data.
>
> A restricted device will act as an initiator only if it becomes a third
> part copy manager (CM) in a third party operation an does copy from one
of
> its devices to another device.
>
> Introducing a new mode bit (as Robert Snively seems to suggest) will not
> change the fact that the restriction can't be upholded
> and do recovery unless the restricted initiator has enough memory.
>
> The spec should only specify a way to terminate a command in those
> conditions and leave it at that.
>
> I will change the wording of the DataOrder to make it clearer but I
> consider the whole issue entirely academic and overblown.
>
> Recall also that a CM implemented which such severe buffering
restrictions
> violates the basic SCSI assumption that a target is the data master.
>
> Regards,
> Julo
>
> Santosh Rao <santoshr@cup.hp.com> on 28/04/2001 00:03:26
>
> Please respond to Santosh Rao <santoshr@cup.hp.com>
>
> To:   Julian Satran/Haifa/IBM@IBMIL
> cc:   Black_David@emc.com, ips@ece.cmu.edu
> Subject:  Re: iSCSI : digest error handling violates EMDP/InDataOrder
>
> julian_satran@il.ibm.com wrote:
> >
> > David,
> >
> > I read Bob's mail and my interpretation is similar to his. However I
> think
> > that SPC explicitly states that different transports are free to
> interpret
> > and make use of this page as they find appropriate.
> >
> > I have a hard time understanding Santosh's objection as it does not
refer
> > to the reason the EMDP is there but to the way it is written in FCP
(not
> > iSCSI).
>
> Julian,
>
> As has been stated earlier, EMDP allows control over the order in which
> the target requests outbound data or sends inbound data. EMDP can be
> used by initiators to control this order and turn off out-of-order R2T
> requests [as well as turn off out of order read data pdus].
>
> This is a useful control option and is already provided by other SCSI
> transports. What good reason exists to deny this provision in iSCSI ?
>
> Also, I have some concerns about the ambiguous definition of DataOrder.
>
> Per the spec :
> "DataOrder=<yes|no>
>
> The default is yes but targets MAY support no. No is used by iSCSI to
> indicate that the data PDUs can be in any order (EMDP = 1). Yes is used
> to indicate that incoming data PDUs have to be at continuously
> increasing addresses (EMDP = 0)."
>
> Based on the above definition wording :
>
> a) How is DataOrder interpreted for WRITE I/Os ?
> b) Is the ordering across the entire SCSI command or a subset of the I/O
> ? If so, what constitutes this subset ?
>
> Different implementors can arrive at different interpretations reading
> the above definition !
>
> - Santosh
>  - santoshr.vcf
 - santoshr.vcf