Re: iSCSI: ERL=1 question.

["Mallikarjun C." <cbm@rose.hp.com>]

Prasenjit,

I tend to agree with you.  

The "must send status after SNACK reject" rule you pointed out was 
originally introduced to avoid infinite loops where an initiator 
continually asks for data re-transmit, and target keeps rebuffing 
(for transient internal reasons, since data SNACK is legal for 
ErrorRecoveryLevel=1).  The intent was to force the I/O completion, 
precluding SNACK retries.

In the case you describe, the target already delivered the status, so
I propose that all we need to ensure is that there are no SNACK retries.
Initiator should be able to internally assume a "Data-SNACK reject"
reason - it does so today anyway if no recovery for the lost data is 
attempted.

Here is the new text for consideration (to replace the bullet (a) text
on page 134 in section 8.4 of rev09) -

a) Request the desired data PDU through SNACK. In its turn,
   the target MUST either resend the data PDU or, reject the 
   SNACK with a Reject PDU with a reason-code of "Data-SNACK 
   Reject" in which case
	(i) if the status had not already been sent for the 
            command, the target MUST terminate the command with 
            an iSCSI command response reason of "SNACK rejected". 
	(ii)if the status was already sent, no further action is
            necessary for the target.  Initiator in this case
            MUST internally signal the completion with the "Data-SNACK 
            reject" reason disregarding any received status PDU, 
            but must wait for the status to be received before doing
            so.
   [OR].....


Note that there's one paranoid case here in sub-bullet (ii) above
where a status SNACK (if status is also lost...) will cause the target 
to report the original status again (SNACK demands a replica) with a 
successful completion.  But in reality, the intervening "Data-SNACK 
Reject" had already sealed the fate of the I/O on the initiator.
The retransmitted status merely fills the StatSN hole and gives the
initiator the determinism to re-use the ITT.

The wording in sub-bullet (ii) will cover this paranoid case as well.

Comments?

To Santosh:

> > upon
> > encountering a datasn hole, say, data_snack_sent, and it should ignore
> > any scsi status received while in this state.

I wouldn't recommend that!  If the Data SNACK were successful, this
status
is the final status for the I/O.  Discarding the status would mean
setting
the stage for a(n unnecessary) status SNACK in future.
-- 
Mallikarjun 


Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
cbm@rose.hp.com


Prasenjit Sarkar wrote:
> 
> Santosh,
> 
> Regarding (B), I did not even venture to imply that there would be a
> stat_sn hole.
> 
> I'm a little confused about your data snack recovery rule, but that is
> besides the point and I can discuss this offline with you.
> 
> My chief concern is that ERL 1 leaves open the possibility for duplicate
> SCSI status PDUs. There are three solutions:
> 
> 1. Leave it to implementors who can use their judgement to decide which
> status to ignore.
> 2. Formulate a tight rule: status PDU with higher stat_sn number always
> wins.
> 3. Change the standard so that duplication does not happen.
> 
> I would like the standard to at least discuss this and then state what path
> they chose and why (with good reason of course). I do not think that this
> is merely an implementation issue.
> 
> My preference is of course: 3,2,1.
> 
>    Prasenjit Sarkar
>    Research Staff Member
>    IBM Almaden Research
>    San Jose
> 
> 
>                     Santosh Rao
>                     <santoshr@cup.       To:     Prasenjit Sarkar/Almaden/IBM@IBMUS
>                     hp.com>              cc:     ips@ece.cmu.edu
>                     Sent by:             Subject:     Re: iSCSI: ERL=1 question.
>                     owner-ips@ece.
>                     cmu.edu
> 
> 
>                     01/03/2002
>                     06:45 PM
> 
> 
> 
> Prasenjit,
> 
> Regarding your question (B)....
> 
> Anytime the initiator's receive path successfully receives a pdu and
> passes format and digest error tests, the local state information of
> exp_cmdsn, max_cmdsn, statsn should be updated based on the values that
> are carried in that pdu and the rules specified in Section 2.2.2.
> 
> Thus, in your below context, when the first scsi status pdu was
> received, the exp_statsn should have gotten updated and there should not
> have been any hole in statsn sequence. What am I missing (?)
> 
> Regarding your question (A)...
> 
> You may have read more into my answer than was intended.
> 
> It is not the job of the lower layered pdu assembly path to filter out
> duplicate scsi status pdu's. Rather, this intelligence would lie within
> the module (finite state machine) that implemented the i/o path logic.
> 
> While a data_snack has been sent and the i/o state machine is awaiting
> missing data pdu's , any received scsi status for that exchange must not
> be sent up to the SCSI ULP. Upon successful receipt of all the missing
> data pdu[s], the status can then be conveyed to the ULP.
> 
> (I don't have a strong opinion either way, if we should have to state
> the above in the draft, or leave it unsaid as implementation specifics.
> I guess, at some point we cross over into the realm of an implementation
> guide vs a protocol specification :)
> 
> - Santosh
> 
> Prasenjit Sarkar wrote:
> >
> > I'm assuming Santosh wanted to reply to the entire group, so I am
> > forwarding his response to the mailing list.
> >
> > Santosh's argument is to provide the (unstated) rule that data assembly
> is
> > the domain of a lower iSCSI layer whose job is to filter out duplicate
> SCSI
> > status PDUs. I would gladly accept this argument except that a standard
> > should present a generic rule which should be independent of how a
> protocol
> > is implemented. I would prefer a rule based on stat_sn ordering or any
> > other equivalent rather than one based on ideal iSCSI layering.
> >
> > Santosh's motivation of having a seamless way of completing commands is
> > probably not true as initiators are allowed to generate SCSI responses.
> >
> >    Prasenjit Sarkar
> >    Research Staff Member
> >    IBM Almaden Research
> >    San Jose
> >
> >
> >                     Santosh Rao
> >                     <santoshr@cup.       To:     Prasenjit
> Sarkar/Almaden/IBM@IBMUS
> >                     hp.com>              cc:
> >                     Sent by:             Subject:     Re: iSCSI: ERL=1
> question.
> >                     santoshr@cup.h
> >                     p.com
> >
> >
> >                     01/03/2002
> >                     05:00 PM
> >
> >
> >
> > Prasenjit,
> >
> > Responses inline.
> >
> > Regards,
> > Santosh
> >
> > Prasenjit Sarkar wrote:
> > >
> > > Assume the following scenario where I and T stand for initiator and
> > target
> > > respectively.
> > >
> > > 1. I->T: Scsi Cmd
> > >
> > > 2. T->I: Scsi Data (DataSN:0)
> > >
> > > 3. T->I: Scsi Status (Good)
> > >
> > > Assume there is a data digest problem for the data with DataSN:0, so
> > >
> > > 4. I->T: Data Snack for DataSN:0
> > >
> > > The target for some reason cannot respond with the data, so according
> to
> > > the spec
> > >
> > > 5: T->I: Reject with reason DATA_SNACK_REJECT
> > >
> > > 6. T->I: Scsi Status (iSCSI response: SNACK rejected -> SCSI READ
> Error)
> > >
> > > The questions are as follows:
> > >
> > > A. Is SAM ambivalent of the fact that there can be two statii for the
> > same
> > > command? (I dont have a problem if SAM doesnt)
> >
> > I'm not certain on what SAM says on this one. However, one can envision
> > that the iscsi i/o finite state machine would enter into some state upon
> > encountering a datasn hole, say, data_snack_sent, and it should ignore
> > any scsi status received while in this state.
> > The lower pdu receive and re-assembly module should have passed format
> > and digest checks (assuming the frame came thru fine) and so, the local
> > value of exp_statsn should have gotten updated within this initiator
> > instance's iscsi connection finite state machine.
> >
> > Thus.....
> > i) since the 1st status was ignored while awaiting a data_snack
> > response, only 1 scsi status ends up being processed by the i/o finite
> > state m/c.
> >
> > ii) there will not be a statsn hole, since the previous status was
> > received successfully by the receive path, resulting in an update to
> > exp_statsn. however, the staus was discarded within the i/o fsm.
> >
> > >
> > > B, Does the second SCSI status have the same StatSN as the first?
> Likely,
> > > it does not, but it should be clearly stated that a SCSI status with
> > higher
> > > stat_sn overrides one with the lower stat_sn.
> >
> > See above.
> >
> > >
> > > C. I'm looking for motivation here: why does the target (rather than
> the
> > > initiator) generate the second status? Couldnt the initiator also do
> the
> > > same on receiving the DATA_SNACK_REJECT?
> >
> > So that all I/Os receive a single method of completion via the SCSI
> > status, I guess.
> >
> > --
> > ##################################
> > Santosh Rao
> > Software Design Engineer,
> > HP-UX iSCSI Driver Team,
> > Hewlett Packard, Cupertino.
> > email : santoshr@cup.hp.com
> > Phone : 408-447-3751
> > ##################################
> 
> --
> ##################################
> Santosh Rao
> Software Design Engineer,
> HP-UX iSCSI Driver Team,
> Hewlett Packard, Cupertino.
> email : santoshr@cup.hp.com
> Phone : 408-447-3751
> ##################################