RE: iSCSI: Out Of Sequence due to null sequence with multiple con nections.

[Black_David@emc.com]

> I would state this much stronger.  Applications had better not have to
know
> that it is iSCSI underneath vs. FCP or parallel SCSI else I believe we
> missed the objective (granted, some things such as target address space
are
> unavoidably different, but I believe task management functions should be
the
> same).  The transport needs to handle the transport issues without
exposing
> quirks to the SCSI or application layer.

Unfortunately, I think we have an impossible situation.  It appears to me
that
we have to pick at most two of the following three goals, as I have yet to
see
any way to achieve all three for a single task management command on a
multiple connection session:

(1) The command takes effect immediately and its status/response
	is available immediately.
(2) The command affects all commands in flight, and its status/response
	is delayed until all such effects are complete.
(3) There is no significant visible departure from existing SCSI task
	management behavior.

The problem is that trying to do both (1) and (2) either requires SCSI to
"execute" the task management command twice or requires that iSCSI do
some task management (e.g., on the in-flight commands) on SCSI's behalf
(or worse like having SCSI prolong the execution of the task management
command until everything in flight in iSCSI arrives).  All of these appear
to lead to problems with (3) in one form or another - two executions
result in two SCSI status/responses that have to be merged, and iSCSI
task management will sooner or later do something different from SCSI
(e.g., I sincerely doubt that a Target in a bridge will ever get this 100%
identical to the devices that are being bridged).

The current iSCSI draft provides the choice of  [(1)] XOR [(2), (3)];
the reason for not getting (3) with (1) is the possibility of the task
management command bypassing commands that it's supposed to
affect.  Charles' original proposal is [(2), (3)] because it has to time out
a stuck connection before executing the command, and is roughly
equivalent to sending the command for ordered delivery and having
the implementation treat any queue between iSCSI and SCSI as
being on the SCSI side of the line.  Doug Otis's counter-proposal
falls into the category of iSCSI doing task management on SCSI's
behalf and provides an example of how this results in visible changes
in behavior -- for the CLEAR ACA task management command,
aborting all tasks that are queued or in flight is generally incorrect.

I would note that this issue does not arise on single connection sessions,
because sending the command for immediate delivery plus some care not
to reorder things in the iSCSI Target (i.e., consider the iSCSI to SCSI
queue
to be in "SCSI" and hence subject to the task management command)
obtains all of (1) through (3).

Going out on a limb, I suspect applications will generally want [(2), (3)]
-- send for ordered delivery and wait for the dust to settle because that
provides the best odds of having some weird device get into a known
state from which further progress is possible.  This allows the application
to not know whether parallel SCSI, FCP or iSCSI is underneath and
relies on other iSCSI recovery procedures to make sure that the task
management command is delivered and executed (e.g., unstick and/or
close "stuck" connections).  There will be cases in which (1) is
needed (e.g., observe tape robot doing something obviously wrong,
and get it to stop immediately), but those may involve fairly blunt
instruments (e.g., LUN RESET) and the need to clean up any collateral
damage.

Sandeep's proposal to create state in the target either fails to achieve
(1) [if the response is delayed until the state is removed] or violates SAM2
[returns the response to the task management command before the task
management command is complete].  Having state linger after a completed
LUN or TARGET RESET is almost certainly wrong.

So, I think I'm down to sending task management functions once, usually
for ordered delivery with the application making the ordered vs. immediate
delivery choice (and sending the task management function twice if it
so chooses).  I think apps will generally choose ordered delivery, choosing
predictable behavior over immediacy concerns.  Aside from a longer
discussion of this issue, I still don't see the need for additional
mechanism(s) to task management - what have I missed in the above
discussion?

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------