RE: iSCSI Requirements Draft - Informal WG Last Call

["Douglas Otis" <dotis@sanlight.net>]

Julian,

I agree that without serialization of commands and then mandating a
sequential delivery scheme, there is no means of ensuring sequential
delivery or allowing atomicity of command groups.  This sequential delivery
also allows rejection of stray commands that may have become trapped within
the network.  A problem of duplicating serialization for iSCSI "immediate"
is that now the receive window can be closed by a "non-immediate" command
carrying the same value.  You will need to adopt connection allegiance for
repeated sequences to avoid a closed window on "immediate" commands.  (It
would also be an improvement if this iSCSI "immediate" term did not overload
the meaning used in SCSI.)

Also, there would appear to be no succession limit imposed on "immediate"
commands.  Santosh will take this freedom to imply all commands can be
treated in this fashion.  If so, terminology that will impose First In First
Out treatment on multiple "immediate" commands that carrying the same
serialization will be needed and connection allegiance seems to provide this
feature.  For succession, perhaps unique serialization re-applies on
subsequent "immediate" commands to allow multiple "immediate" commands to be
serialized across multiple connections and to re-enable iSCSI flow control.
The problem this creates results in apparent skips for the sequencer
tracking only "non-immediate" commands.  (Rejecting commands bypassed is one
method to handle this problem.)  There would not need to be a
"non-immediate" command to cause this increment in the case of no
"non-immediate" commands ever presented.  This implies however two
sequencers are operating.  One for "immediate" and another for
"non-immediate."  In essence, not using "immediate" would be the same as
only using "immediate."  The "Casual" ordering sought by Santosh does not
seem possible within these constraints without an additional flag to
explicitly allow such treatment.  There would then need to be warnings about
SCSI Task Full flow control used in lieu of the iSCSI scheme.  How do you
envision the use of "immediate"?  The proposal does not provide much
clarification about potential limits on successive use.

The ability to flag these commands by the driver is left in doubt by this
proposal and termed out of scope.  Would it be prudent to adopt a scheme
where either Head of Queue or ACA tasks are marked for this treatment?
Depending on the implementation size of the buffers feeding the SCSI layer
on the target side, those commands bypassed can be problematic.  If the
definition of SCSI layer is to imply yet another queue where all commands
are held in sequence regardless of this iSCSI "immediate" flag, then
solutions are limited but then the function of the "immediate" flag is then
severely reduced as well.  If the "immediate" flag is still used within the
SCSI layer, then the definition of SCSI layer is a bit odd and should be
clarified.

You may wish to elaborate what you expect to be done in the case where tasks
to be acted upon are bypassed.  In my opinion, rejecting these commands back
to the initiator is the corrective action.  This rejection scheme also
allows "immediate" commands to carry their own serialization to benefit from
iSCSI flow control remaining valid, connection allegiance not a concern for
the initiator, the command window immediately opening, and "immediate"
commands then receive acknowledgement.  The only provision for rejection
would be that prior "immediate" commands would not be rejected back to the
initiator.

Ver 5-91, Pg 11,
   "If immediate delivery is used with task management commands, these
   commands may reach the SCSI target task management before the tasks
   they are supposed to act upon.  However, their CmdSN is a good
   reference for what commands the immediate command was supposed to act
   upon."

   "The target MUST NOT transmit a MaxCmdSN that is more than 2**31 - 1
   above the last ExpCmdSN.  For non-immediate commands, the CmdSN field
   can take any value from ExpCmdSN to MaxCmdSN. For immediate commands,
   the CmdSN field can take any value from ExpCmdSN to MaxCmdSN+1. The
   target MUST silently ignore any command outside this range or
   duplicates within the range that have not been flagged with the retry
   bit (the X bit in the opcode)."


Doug

> Ordered delivery of commands to ANY TYPE of devices will increase in
> importance as network speeds increase and the need to hide latency
> increases.
>
> Today databases don't use queuing and rely and trickle the commands to
> devices 1 by 1 to ensure atomicity and order.
> As latency will become the determining factor in performance this is bound
> to change.
>
> SCSI has done an excellent job in defining the queueing mechanism. We have
> to make it work with good performance in our environment.
>
>
>
> Julo
>
> Santosh Rao <santoshr@cup.hp.com> on 13/04/2001 04:33:45
>
> Please respond to Santosh Rao <santoshr@cup.hp.com>
>
> To:   ips@ece.cmu.edu
> cc:   Black_David@emc.com
> Subject:  Re: iSCSI Requirements Draft - Informal WG Last Call
>
>
>
>
> David & All,
>
> I object to the following requirement :
>
> " MUST support ordered delivery of SCSI commands from the initiator to
> the
>   target, to support SCSI Task Queuing. "
>
> Ordered delivery is not a requirement for disk based applications and
> non tagged queueing tape applications, which form the majority of
> today's data traffic.
>
> To impose strict ordering (even in the presence of errors ?) as a MUST
> is penalizing the majority of today's data traffic that does not expect
> ordering from the SCSI subsystem.
>
> I am particularly concerned about the effect of the above requirement in
> the presence of errors. Does iSCSI expect strict ordering to be
> maintained even when individual I/O errors like ULP timeout occur ?
>
> On a ULP timeout (caused by, say, a hole in CmdSN), the initiator may
> choose not to retry the command, but instead, error it back to the ULP.
> In such a case, it can plug the hole in CmdSN with a NOP-OUT.
>
> The above requirement is not feasible to be met under such circumstances
> and others similar to this. Mandating strict ordering on ULP timeouts
> implies a session level error recovery on any individual I/O being
> failed back from iSCSI to SCSI ULP. This is a very heavy hammer to use
> as error recovery and should not be imposed.
>
> The above requirement must be changed to :
> " SHOULD support ordered delivery of SCSI commands from the initiator to
> the
>   target, to support SCSI Task Queuing. "
>
> - Santosh
>
>
>
> Black_David@emc.com wrote:
> >
> > It is intended to submit draft-ietf-ips-iscsi-reqmts-02.txt
> > as an Informational RFC. There is no formal requirement for
> > a WG Last Call, but if you have any further substantive comments
> > on the document please raise them on this list within the next
> > two weeks, i.e. by April 27th at the latest.
> >
> > If you have typographical/editorial comments please send them
> > direct to the document's author, Marjorie Krueger
> > <marjorie_krueger@hp.com>.
> >
> > Thanks,
> > --David and Elizabeth, IPS WG co-chairs
>  - santoshr.vcf
>
>
>
>