RE: iSCSI: Header Digest Failure (was R2TDataSN)

[Douglas Otis <dotis@sanlight.net>]

Julian,

It should follow that frame markers are not an option if header digests are
used otherwise dropping the connection becomes required in the event of an
error.  If header digests are used, there should also be an advisory
recommending checking both values of the frame marker PDU pointer to detect
a potential error at this level, as well as, a recommended null stuffing of
the connection to allow resynchronization between framing markers.  Here in
the draft, you have overlooked a concern regarding a digest error as it
pertains to framing.

Page 20:
   "In situations where IP packets are delivered in order from the
   network, iSCSI message framing is not an issue; messages are
   processed one after the other. In the presence of IP packet
   reordering (e.g., frames being dropped), legacy TCP implementations
   store the "out of order" TCP segments in temporary buffers until the
   missing TCP segments arrive, upon which the data must be copied to
   the application buffers.  In iSCSI it is desirable to steer the SCSI
   data within these out of order TCP segments into the pre-allocated
   SCSI buffers rather than store them in temporary buffers. This
   decreases the need for dedicated reassembly buffers as well as the
   latency and bandwidth related to extra copies."


Page 12:

   "The iSCSI target layer MUST deliver the commands to the SCSI target
   layer in the order specified by CmdSN."

What happens if there is an unrecovered gap in the CmdSN due to a failed
NIC?  Why are these 31 bits? (2**31-1) is the limit set for several values.
This is 2,147,483,647.  In other words, this is a 31 bit limitation.  If the
MS bit is not to be used, perhaps this bit could be used to signal Immediate
without disrupting the command sequence.  I doubt this is intended however
as there are examples that indicate a count at 0xffffffff.  If the MS bit
was not used, then the maximum count would have been 0x7fffffff.  I hope I
am not repeating a prior notice.

Page 18:
      - IPv6 address, in dotted decimal notation.  Assumed if the
      name contains more than four, but at most 16 numbers, separated
      by dots (.), where each number is in the range 0..255.

Staying with decimal byte notation for IPv6, define how a value with fewer
than 16 digits is interpreted by means of padding.  Do you zero pad MS
positions?  This will require scanning all values with reversed order
placement.  I doubt there is a more expensive scheme.

On page 25, perhaps a better choice of words would be 'section' rather than
'segment' to describe portions of the various structures that comprise the
PDU layers.

On page 83, you go to some length explaining R2T recovery based on a timeout
but fail to provide any indication of a suitable timeout value.  How are
these timeouts negotiated?
Can equipment be expected to interoperate without timeouts specified?

Doug


> I understood that. But markers are meant to solve this too aren't they?
>
> Julo
>
> Matt Wakeley <matt_wakeley@agilent.com> on 08/03/2001 01:54:10
>
> Please respond to Matt Wakeley <matt_wakeley@agilent.com>
>
> To:   ips@ece.cmu.edu
> cc:
> Subject:  iSCSI: Header Digest Failure (was R2TDataSN)
>
>
>
>
> Julian,
>
> What Somesh is talking about has nothing to do with R2T specifically.  His
> point is, the length field in the PDU header determines how long
> *this* PDU
> is, and where to find the *next* PDU.  If *this* PDU fails its header
> digest
> check, you cannot trust the length field, therefore, you cannot know where
> the
> *next* PDU is in the byte stream.
>
> Given that, is seems like whenever there is a header digest
> error, you have
> no
> choice but to terminate the TCP connection and start a new one.
>
> -Matt Wakeley
> Agilent Technologies
>
>
> Somesh Gupta wrote:
> >
> > Julian,
> >
> > How do I know it is an R2T if the header digest failed?
> > I am sorry if I am missing something very obvious here.
> >
> > Let us say that a header digest failed. What information
> > in the header am I able to trust at this point? I don't
> > know if it is an R2T or not. I don't know if the length
> > is ok or not.
> >
> > Thanks,
> > Somesh
> >
> > > -----Original Message-----
> > > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> > > julian_satran@il.ibm.com
> > > Sent: Tuesday, March 06, 2001 4:01 PM
> > > To: ips@ece.cmu.edu
> > > Subject: RE: R2TDataSN
> > >
> > >
> > >
> > >
> > > Somesh,
> > >
> > >  I am still lost. R2T is fixed length and you know that you have a bad
> > > digest after reading it.
> > > No length involved.   ???
> > >
> > > Julo
> > >
> > > "Somesh Gupta" <someshg@yahoo.com> on 07/03/2001 00:26:10
> > >
> > > Please respond to someshg@yahoo.com
> > >
> > > To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
> > > cc:
> > > Subject:  RE: R2TDataSN
> > >
> > >
> > >
> > >
> > > Julian,
> > >
> > > > -----Original Message-----
> > > > From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf
> Of
> > > > julian_satran@il.ibm.com
> > > > Sent: Tuesday, March 06, 2001 9:31 AM
> > > > To: ips@ece.cmu.edu
> > > > Subject: RE: R2TDataSN
> > > >
> > > >
> > > >
> > > >
> > > > Somesh,
> > > >
> > > > You've lost me. I do not propose that you look at the bad
> R2tT but to
> > > find
> > > > that you have missed one
> > > > by looking at the next.
> > >
> > > Since iSCSI PDUs define how long they are, you have to look at
> > > one PDU to determine where the next PDU is. (unless ofcourse
> > > the sync and steering layer is doing the work - see my exchange
> > > with Venkat on that).
> > >
> > > On the long transfer etc., I was not sure what scenarios
> > > an R2TDataSN was providing recovery from. Since you clarified
> > > that it is to recover from a header digest error, we can
> > > focus on that scenario.
> > >
> > >
> > > This interesting for long transfers that have
> > > > several outstanding R2Ts.
> > > > What is speculative here? There was never a consensus that there
> > > > will be no
> > > > more than one outstanding R2T.
> > > >
> > > > Regards,
> > > > Julo
> > > >
> > > > "Somesh Gupta" <someshg@yahoo.com> on 06/03/2001 17:23:04
> > > >
> > > > Please respond to someshg@yahoo.com
> > > >
> > > > To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
> > > > cc:
> > > > Subject:  RE: R2TDataSN
> > > >
> > > >
> > > >
> > > >
> > > > I beg to disagree. If an R2T PDU (header) has bad digest, or any
> other
> > > > header has a bad digest - since you always need the PDU length from
> > > > the header, there is some uncertainty associated with further
> > > processing.
> > > >
> > > > Are you proposing that the processing machine go into a
> "speculative"
> > > > mode where the processing of the next PDU determines whether we were
> > > > successfuly able to skip a bad PDU header? When there is a data
> digest
> > > > error, further stream parsing is deterministic. But not when the PDU
> > > > header digest error.
> > > >
> > > > Also the consensus (in my interpretation) was on applications
> > > > not transfering very large amounts of data using a single command or
> > > > read/write PDU.
> > > >
> > > > > -----Original Message-----
> > > > > From: owner-ips@ece.cmu.edu
> [mailto:owner-ips@ece.cmu.edu]On Behalf
> Of
> > > > > julian_satran@il.ibm.com
> > > > > Sent: Monday, March 05, 2001 5:41 PM
> > > > > To: ips@ece.cmu.edu
> > > > > Subject: Re: R2TDataSN
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Somesh,
> > > > >
> > > > > 1.The only consensus I heard is not to transfer a large amount of
> > > > > data with
> > > > > one PDU.
> > > > >
> > > > > 2.With DatasN and Sack we dont need any data in a bad header.
> > > > >
> > > > > 3. If an R2T is lost (received at initiator with bad digest) - the
> > > > > initiator will know that from
> > > > > the next R2T if the target has several outstanding - very
> likely at
> > > long
> > > > > distances - and will not have to way for a timeout.   Other uses
> are
> > > > > marginal.  Basically it is "part of a command execution"
> and we can
> > > > > painless recover
> > > > > from failures for this case too.
> > > > >
> > > > > Regards,
> > > > > Julo
> > > > >
> > > > > "Somesh Gupta" <someshg@yahoo.com> on 05/03/2001 20:40:06
> > > > >
> > > > > Please respond to someshg@yahoo.com
> > > > >
> > > > > To:   Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
> > > > > cc:
> > > > > Subject:  R2TDataSN
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > R2TDataSN
> > > > > > ----------
> > > > > > Sec 6.7.1 has some new content on how to handle lost R2Ts using
> > > > > > SACKs.  But I noticed that the SACK request (Sec 2.16) has not
> > > > > > changed to indicate whether the DataSN is a R2T DataSN or just
> > > > > > a Read PDU DataSN (D bit)
> > > > > > So do we demux on the read/write operation type?
> > > > > > And how does this affect PDU loss in bidirectional commands ?
> > > > > > +++ SACK is ascking for data (DataSN) the target knows
> > > > > >
> > > > >
> > > > > Julian,
> > > > >
> > > > > Regarding the R2TDataSN, I have a comments and a
> > > > > question.
> > > > >
> > > > > I think that when a PDU header fails a CRC/checksum check etc,
> > > > > it is a problem to depend on any information in the header
> (including
> > > > > length fields), thereby making any further processing on
> > > > > the connection unreliable.
> > > > >
> > > > > What scenarios do you envision where the R2TDataSN is useful.
> > > > > IN Orlando I think there was clear consensus that application
> > > > > do not try to transfer very large amounts of data using a
> > > > > single command.
> > > > >
> > > > > Thanks,
> > > > > Somesh
> > > > >
> > >
> > > Thanks,
> > > Somesh Gupta
> > >
> > >
> > > _________________________________________________________
> > > Do You Yahoo!?
> > > Get your free @yahoo.com address at http://mail.yahoo.com
> > >
> > >
> > >
> >
> > _________________________________________________________
> > Do You Yahoo!?
> > Get your free @yahoo.com address at http://mail.yahoo.com
>
>
>
>
>