RE: iSCSI Reqts: In-Order Delivery

["Douglas Otis" <dotis@sanlight.net>]

David,

In my prior message:
http://ips.pdl.cs.cmu.edu/mail/msg04218.html

I included a concern with respect to the in-order delivery of Server
responses.  In particular, it concerns Management responses with respect to
prior Server responses.  With multiple connections, the per-connection
serialization from the Server does not allow any assurances with respect to
sequential delivery of responses.  This may cause unexpected return of
status assumed encompassed by a management response affected by Management
requests sent on different connections.

In general, the Must requirement seems appropriate.  There may be some gray
area with respect to SCSI layer delivery.  Admittedly, I have not full
reviewed the latest version of the iSCSI specifications.

Doug


> About a week ago, Santosh Rao wrote:
>
> > I object to the following requirement :
> >
> > " MUST support ordered delivery of SCSI commands from the initiator to
> > the  target, to support SCSI Task Queuing. "
>
> In the hopes of driving this discussion to closure,
> let me summarize my understanding of this situation:
>
> - SAM2 expects ordered delivery, but does not mandate
> 	it.  Section 4.6.2 of SAM2r16 says:
>
>   For convenience, the SCSI architecture model assumes in-order delivery
>   to be a property of the service delivery subsystem. This assumption is
>   made to simplify the description of behavior and does not constitute a
>   requirement.
>
> 	I will observe that there is well-known code for disk
> 	initiators (e.g., staircase/elevator/etc. command queue
> 	reordering for seek optimization) that relies on this
> 	expectation for performance (not correctness).  IMHO,
> 	making in-order delivery purely OPTIONAL would not be
> 	consistent with the spirit/intent of the above SAM2 text.
>
> - In existing SCSI transports, transport errors (e.g., CRC failure)
> 	can cause ordering issues at higher levels.  Both Parallel SCSI
> 	and Fibre Channel initially drop anything that has a transport
> 	error.  IF the higher level code retries AND other commands were
> 	executed before the error was noticed by the retry logic, THEN
> 	the retry occurs in a different place in the command sequence
> 	from the original command.  I believe that existing tape
> 	implementations usually either don't retry, or spoon-feed the
> 	tape one command at the time to make sure no other command can
> 	get in the way.  FCP-2 has the ability to prevent other commands
> 	from being executed ...
>
> - iSCSI has to face this issue because we're incorporating a retry
> 	mechanism into the base protocol.  Parallel SCSI and FCP-1 could
> 	ignore this because they didn't do retries.  FCP-2 has a retry
> 	mechanism, but makes order preservation of retries negotiable -
> 	order of command delivery in the face of retries is only assured
> 	if CRN is in use, and that is controlled by a settable bit in a
> 	mode page (FCP2r7, 4.3).  The Initiator can choose not to set
> 	this bit, and the Target can refuse to allow the Initiator to set
> 	it.
>
> - CRN is a relatively new mechanism, and both it and the associated
> 	FCP-2 retry mechanism are likely to be implemented for/used by
> 	tapes.  Disks will still have an expectation of in-order delivery
> 	in some cases, even though neither of these mechanisms are in use.
> 	IMHO, Santosh's proposal to use CRN for all
> expectations/requirements
> 	of ordering doesn't seem like a good idea for this reason.
>
> So, let me try the following proposal to resolve this issue/objection:
>
> (1) MUST provide ordered delivery of SCSI commands from
> 	the initiator to the target in the absence of transport
> 	errors visible to iSCSI (e.g., iSCSI CRC failure,
> 	unexpected TCP connection closure).
> (2) MUST specify the ability to preserve ordered delivery
> 	of SCSI commands even in the presence of transport
> 	errors.  A mechanism MUST be provided to allow
> 	Initiators and Targets to negotiate this preservation
> 	on a per-session or finer granularity basis,
>
> The Rationale for (1) is the combination of the SAM2 expectation
> plus the fact that there are situations in which disks expect
> this ordering in the absence of mechanisms like CRN that can
> enforce it.  The Rationale for (2) is to provide support
> analogous to FCP-2 - this should be sufficient for tapes to
> obtain the behavior they require.
>
> Comments?
>
> Thanks,
> --David
>
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
> black_david@emc.com       Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>