RE: iSCSI: Recognizing Recovery R2Ts

["Julian Satran" <Julian_Satran@il.ibm.com>]

Nick,

I understand that your issue is mainly related to an initiator expecting "in order" R2Ts and seing an R2T that is out of order.

The simplest answer (and the one that we had in mind when we designed the protocol) is that in general nodes willing to accept ErrorRecoveryLevels 1 and 2 will probably accept R2Ts out of order. In this case an initiator does not need to make any distinction.

But not wanting to be that restrictive we let even nodes accepting only ordered R2Ts do recovery but then (if they care) they may::

• initiators:
  

1. reject any R2T that is unordered (i.e. do not recover)
2. accept R2T based on having "previously seen it"  (as David Black suggests) - with the caveat that if the R2T is lost you may have to fall back to  1

• targets:

1. have only one R2T outstanding if you have both "in order" and error recovery (a rare thing we believe)
   
   
   Regards,
   Julo
   
   
   

   Black_David@emc.com Sent by: owner-ips@ece.cmu.edu 16/09/02 04:58

   To:        nickb@attheoffice.org         cc:        ips@ece.cmu.edu         Subject:        RE: iSCSI: Recognizing Recovery R2Ts

   
   
   One additional important note - the Recovery R2T generates
   its own data sequence, so in the following:
   
   > > Following a)
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=16384 DataSN=2, DataLength=8192
   > > I-> ISCSI_INIT_SCSI_DATA_OUT F_BIT, Offset=24576, DataSN=3,
   > > DataLength=8192
   > >
   > > Following B)
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=16384 DataSN=2, DataLength=8192
   
   The DataSN's should be 0 and 1, not 2 and 3 (and in the B) case above,
   the F bit needs to be set, as I previously noted).  The initiator really
   does respond to a Recovery R2T in the same fashion as any other R2T.
   
   Thanks,
   --David
   
   > -----Original Message-----
   > From: Black_David@emc.com [mailto:Black_David@emc.com]
   > Sent: Sunday, September 15, 2002 9:45 PM
   > To: nickb@attheoffice.org
   > Cc: ips@ece.cmu.edu
   > Subject: RE: iSCSI: Recognizing Recovery R2Ts
   >
   >
   > > Im still lost here.  Please consider the example:
   > >
   > > MaxDataSegmentLength=8192, MaxBurstLength=32768, ImmediateData=No,
   > > InitialR2T=Yes.
   > >
   > > I-> ISCSI_INIT_SCSI_CMND, EDTL=131072, SCSI Opcode WRITE_10
   > > T-> ISCSI_TARG_R2T, DDTL=32768, Offset=0
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=0, DataSN=0, DataLength=8192
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=8192, DataSN=1, DataLength=8192
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=16384, DataSN=2,
   > DataLength=8192
   > >
   > > (Data Payload Fails on DataSN=2)
   > >
   > > T-> ISCSI_TARG_RJT, Reason=Data Digest (Payload) Error
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, F_BIT, Offset=24576, DataSN=3,
   > > DataLength=8192
   > >
   > > Can the Target either: (Are both of these legal?)
   > >
   > >                  a) Accept DataSN=3 even though DataSN=2 Failed
   > >                     Only request retransmisson of Failed DataOut
   > >                  T-> ISCSI_TARG_R2T (Recovery) DDTL=8192, Offset=16384
   > >
   > >                  or b) Discard DataSN=3, Request retransmission of
   > >                        all DataOut from failed to MaxBurstLength  
   > >                  T-> ISCSI_TARG_R2T (Recovery) DDTL=16384, Offset=16384
   >
   > Yes, both are legal, a) is preferred for obvious reasons.
   >
   > > Initiator: (This is where im confused)
   > >                  Received either a) or b),  and knows the R2T is a Recovery
   > >                  R2T because the DDTL is not 32768 (MaxBurstLength or rest of
   > >                  the original EDTL) and Offset is not 32768.
   >
   > Not exactly.  The R2T is a recovery R2T because it requests data
   > covered by the previous R2T (Offset < 32768).  This doesn't affect
   > the initiator's behavior - it just responds to the R2T it receives.
   >
   > > Following a)
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=16384 DataSN=2, DataLength=8192
   > > I-> ISCSI_INIT_SCSI_DATA_OUT F_BIT, Offset=24576, DataSN=3,
   > > DataLength=8192
   > >
   > > Following B)
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=16384 DataSN=2, DataLength=8192
   >
   > I think you flipped the a) and b) cases above, so you clearly are
   > confused :-).  In both cases, the initiator looks at the R2T to
   > determine what data the Target is asking for and sends it - a) asked
   > for 8k, b) asked for 16k.  The one PDU that responds to a) needs
   > to have the F bit set, as it completes the response to the second
   > R2T.  If the target needs to distinguish the recovery data transfer
   > from the original transfer, it should use a different Target Transfer
   > Tag in the recovery R2T.
   >
   > > Target:
   > >                  MaxBurstLength is Completed:
   > > T-> ISCSI_TARG_R2T, Offset=32768, DDTL=32768
   > >
   > > I-> ISCSI_INIT_SCSI_DATA_OUT, Offset=32768, DataSN=0,
   > DataLength=8192
   >
   > Ok.
   >  
   > > Your statement "Whether the initiator knows this depends on
   > whether it
   > > saw the previous R2T" is still confusing to me.  The logic
   > used in the
   > > above example (EDTL and Offset) I assume is incorrect.  
   > Would you mind
   > > elaborating with a specific example of the proper method of
   > doing so?
   >
   > I don't understand the problem here.  In all cases, the initiator
   > looks at the offset and length in the R2T it receives and sends the
   > data requested by that R2T.  In both cases a) and b) above,
   > the initiator
   > can tell that it's a recovery R2T because the requested data
   > range overlaps
   > a previous R2T.  The statement I wrote covers a completely different
   > case in which the target sends an R2T that doesn't arrive
   > (e.g., header
   > digest error), and decides to try to recover from that.
   >
   > I also don't understand the reference to EDTL in the command
   > (as opposed to DDTL in the R2T0.  If a Recovery R2T causes
   > more than EDTL of data to be transferred for a command that's fine -
   > the target is in charge of data flow and will tell the initiator when
   > the command was complete and how much was actually transferred
   > (consider unexpected end-of-tape on a tape device to understand
   > why this is done this way).
   >
   > Thanks,
   > --David
   > ---------------------------------------------------
   > David L. Black, Senior Technologist
   > EMC Corporation, 42 South St., Hopkinton, MA  01748
   > +1 (508) 249-6449            FAX: +1 (508) 497-8018
   > black_david@emc.com       Mobile: +1 (978) 394-7754
   > ---------------------------------------------------
   >