RE: iSCSI: NOPs should not have a CmdSN

["Julian Satran" <Julian_Satran@il.ibm.com>]

The need to reject immediate commands came up several times during recovery
discussions.
NOP with CmdsN was there to allow checking the "delivery machine"
end-to-end.
The text for CmdSN in case of retry had also some points that needed
clarification.
Here is the (more liberal)  text for numbering  commands:


1.1.1.1   Command Numbering and Acknowledging

   iSCSI supports ordered command delivery within a session.  All commands
   (initiator-to-target PDUs) are numbered.

   Any SCSI activity is related to a task (SAM-2). The task is identified
   by the Initiator Task Tag for the life of the task.

   Commands in transit from the initiator to the target layer are numbered
   by iSCSI; the number is carried by the iSCSI PDU as CmdSN
   (Command-Sequence-Number).  The numbering is session-wide.  All outgoing
   iSCSI PDUs that have a task association, except Data-Out, carry this
   number. CmdSNs are allocated by the initiator iSCSI within a 32-bit
   unsigned counter (modulo 2**32). Comparisons and arithmetic on CmdSN
   SHOULD use Serial Number Arithmetic as defined in [RFC1982] where
   SERIAL_BITS = 32.

   Commands meant for immediate delivery are marked as such through an
   immediate delivery flag. They MAY carry any CmdSN. The CmdSN is not
   advanced for commands marked for immediate delivery.

   If immediate delivery is used with task management commands, these
   commands may reach the SCSI target task management before the tasks they
   are supposed to act upon.  However, their CmdSN is a good reference for
   what commands the immediate command was supposed to act upon; to achieve
   this function the task management command MUST be the same CmdSN as that
   which would have been given to the next non-immediate command.

   Not covered in this document are the means by which one may request
   immediate delivery for a command or by which iSCSI will decide by itself
   to mark a PDU for immediate delivery.

   Please note that the number of commands used for immediate delivery is
   not limited and their delivery to execution is not acknowledged through
   the numbering scheme.  Immediate commands can be rejected by the iSCSI
   target due to lack of resources. An iSCSI target MUST be able to handle
   at least one immediate task management command and one immediate
   non-task-management iSCSI request at any time.

   Except for the commands marked for immediate delivery the iSCSI target
   layer MUST deliver the commands for execution in the order specified by
   CmdSN. Commands marked for immediate delivery may be handed over by the
   iSCSI target layer for execution as soon as detected. iSCSI may avoid
   delivering some command for execution if so required by some prior SCSI
   or iSCSI action (e.g., clear task set Task Management request received
   before all the commands it was supposed to act on).

   The initiator and target are assumed to have three registers, unique
   session wide, that define the numbering mechanism:

       - CmdSN - the current command Sequence Number advanced by 1 on each
      command shipped except for commands marked for immediate delivery.
       - ExpCmdSN - the next expected command by the target. The target
      acknowledges all commands up to but not including this number. The
      target iSCSI layer sets the ExpCmdSN to the largest non-immediate
      CmdSN that it is able to deliver for execution plus 1 (no holes in
      the CmdSN sequence).
       - MaxCmdSN - the maximum number to be shipped. The queuing capacity
      of the receiving iSCSI layer is MaxCmdSN - ExpCmdSN + 1.

   ExpCmdSN and MaxCmdSN are derived from target-to-initiator PDU fields.

   MaxCmdSN and ExpCmdSN fields are processed as follows:

      -if the PDU MaxCmdSN is less than the PDU ExpCmdSN-1 (in Serial
      Arithmetic Sense and with a difference bounded by 2**31-1), they are
      both ignored
      -if the PDU MaxCmdSN is less than the local MaxCmdSN (in Serial
      Arithmetic Sense and with a difference bounded by 2**31-1), it is
      ignored; else it updates the local MaxCmdSN
      -if the PDU ExpCmdSN is less than the local ExpCmdSN (in Serial
      Arithmetic Sense and with a difference bounded by 2**31-1), it is
      ignored; else it updates the local ExpCmdSN

   This sequence is required as updates may arrive out of order because
   they travel on different TCP connections.

   The target MUST NOT transmit a MaxCmdSN that is more than 2**31 - 1
   above the last ExpCmdSN.  For non-immediate commands, the CmdSN field
   can take any value from ExpCmdSN to MaxCmdSN. The target MUST silently
   ignore any non-immediate command outside this range or non-immediate
   duplicates within the range that have not been flagged with the retry
   bit (the X bit in the opcode).

   iSCSI initiators and targets MUST support the command numbering scheme.

   A numbered iSCSI command will not change its allocated CmdSN regardless
   of the number of times and circumstances in which it is reissued.  At
   the target, it is assumed that CmdSN is relevant only while the command
   has not created any execution state (can't find the Initiator Task Tag).
   Afterwards CmdSN becomes irrelevant.  Testing for execution state is
   assumed to precede any other action at the target and is followed by
   ordering and delivery if no execution state is found or delivery if
   execution state is found. Immediate commands can't be retried unless
   there is execution state available at the target for them (they are
   rejected for retry if the target can't find the Initiator Task Tag).


   Julo