RE: iSCSI: Out Of Sequence due to null sequence with multipleconnections.

["Douglas Otis" <dotis@sanlight.net>]

Julian,

It could be that a "stuck" command in flight or at the sequencer is the
reason for the task management which is made problematic by the null
sequence.  Even if implementers are aware of this problem, there is not a
good solution until null sequence is removed.  Making a flag to indicate
"immediate" or perhaps "reject prior pending iSCSI commands" is a means of
ensuring the initiator remains in control with respect to all situations.
This ensures the initiator is aware of the state of the target and sequence
of commands can be maintained if required.  Be careful about being too disk
centric.  Treating these SN numbers as unsigned allows a simple means of
tracking.  Here is an example explaination:

   Comparisons and arithmetic on SNs in this document SHOULD use Serial
   Number Arithmetic as defined in [RFC1982] where SERIAL_BITS = 32.

Doug

> David,
>
> Your summary is correct. And (except for a minor point) it is all a matter
> of target implementation.
> SCSI is not a "completely layered" stack and others have gone so far as to
> do task cancellation by an action at link layer (parallel and fiber).
> There might be 2 funny side-effects though if an implementer chooses to
> cancel "holes" (commands in flight on other connections):
>
> 1)the cancelled command is a another task management command (there can be
> only one active but what it the one active gets stucked?)
> 2)(academic I admit) the cancelled command arrives after a wrap around in
> command sequencing; this is a bit harder (although not impossible) to fix
> in the implementation
>
> Implementers should be aware of those side effects.
>
> Regards,
> Julo
>
>
>
> Black_David@emc.com on 26/03/2001 18:57:48
>
> Please respond to Black_David@emc.com
>
> To:   dotis@sanlight.net
> cc:   ips@ece.cmu.edu
> Subject:  RE: iSCSI: Out Of Sequence due to null sequence with
> multipleconn
>       ections.
>
>
>
>
> Let me apologize for unintentionally stepping on Doug
> in the meeting.  Due to the time squeeze, I neglected
> to ask for other issues at the end of the iSCSI discussion
> - sorry about that.
>
> I'm going to back off and try to take a high level view of
> this and see what sort of observations emerge.  When a SCSI
> task management command pops out of an iSCSI TCP connection
> at the target, there are four places that the SCSI operations
> it affects could be:
>
> (1) Executing in SCSI.
> (2) Queued to SCSI for execution, but not executing.
> (3) Queued in iSCSI waiting for command sequencing.
> (4) In-flight.
>
> (2) includes the "resource limitations between the
> sequencer and the target that may lead to a stall or
> a long term delay".
>
> (1) and (2) are the easy cases - the SCSI implementation
> must apply the task management command to executing tasks,
> and should perform the obvious "peephole optimization" to
> the commands queued for execution (i.e., if they're to be
> aborted, abort them and send the response(s) without starting
> execution).  In essence, this models a command as crossing
> the boundary from iSCSI to SCSI the moment that iSCSI is
> prepared to give it to SCSI (i.e., any queue and related
> resource limitations are on SCSI's side of the line).
>
> (4) is hard.  One SCSI task management command generates one
> response.  That response can either be generated immediately
> (command arrives, is passed to SCSI, SCSI does its thing) or
> at the right point in the sequence (command arrives, is
> sequenced by iSCSI, passed to SCSI at the right point in the
> sequence, and SCSI does its thing), but NOT both.  As things
> currently stand, having a task management command apply to
> in-flight commands requires sending the task management
> command for ordered delivery - so if it's desired to have
> the task management command take immediate effect and also
> catch everything in flight, it's going to have to be sent
> twice.  I'm not enthusiastic about the idea of the task
> management command taking immediate effect but delaying the
> response until everything in flight that might be affected
> arrives, as I suspect the Initiator would like to know what
> happened sooner rather than later.
>
> (3) is "interesting".  The results of applying a SCSI task
> management command to a SCSI operation are known only to
> SCSI, and hence asking that a command stuck in the iSCSI
> sequencer be affected immediately by a task management
> command is asking that the task management command have
> the side effect of changing some of the commands it affects
> to immediate delivery so that it can immediately do its
> (SCSI) thing to them.  I wouldn't want to mandate this,
> nor would I want to prohibit it, BUT ... if the above
> discussion of in-flight commands is correct, I would
> observe that the application on the Initiator side
> can't tell the difference between commands that are in-flight
> vs. waiting for something in-flight on another connection,
> and hence is going to have to issue the task management
> command for ordered delivery if it wants to affect operations
> in either place (and issue a second copy if it wants
> immediate action).
>
> The upshot is that, aside from a longer discussion of this
> issue, I'm not sure anything needs to be changed.  Comments?
>
> Thanks,
> --David
>
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> black_david@emc.com       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>
>
>
>