RE: iSCSI: items discussed at WG meeting

["Dave Peterson" <dap@cisco.com>]

2nd try, havn't seen first post...

Howdy Santosh,
Comments on your comments below...
Dave

> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Santosh Rao
> Sent: Wednesday, March 20, 2002 6:20 PM
> To: Dave Peterson
> Cc: Ips@Ece. Cmu. Edu
> Subject: Re: iSCSI: items discussed at WG meeting
>
>
> Dave,
>
> Some queries inline.
>
> Regards,
> Santosh
>
> Dave Peterson wrote:
>
>
> > 4. Clause 6.7 SCSI Timeouts: explicitly state that a command
> retry shouldn't
> > be performed after a SCSI level timeout.
> >
> > "An iSCSI initiator MAY attempt to plug a command sequence gap
> on the target
> > end (in the absence of an acknowledgement of the command by way
> of ExpCmdSN)
> > before the ULP timeout by retrying the unacknowledged command,
> as described
> > in Section 6.1 Retry and Reassign in Recovery."
>
> Why would a scsi transport protocol be required to specify this ? Since
> a ULP timeout is O.S. specific and is not an externally observable
> value, either on the wire, or communicated to the target, why should
> iscsi specify this ?
>
> Different O.S. drivers may apply different timeout policies, with some
> drivers allowing for some grace time to attempt to complete the I/O
> after a timeout.
>
> IMO, the iscsi spec can be silent on what an O.S. specific driver does
> when its O.S. specific ULP timeout expires. This is O.S. dependent.
>
The point is that iSCSI command retry (not an OS retry) does not work
following an abort.

>
> > 6. Text stating that a TMF=Abort Task must be issued for each
> outstanding
> > command. (Clause 6.7)
>
> What does this mean (?). Could you elaborate further on this. 6.7
> defines what action should be taken by an initiator on a SCSI timeout,
> which occurs on a per-command basis and is dealt with individually for
> each command.
>
> Where do the remaining outstanding commands come into the picture,
> unless, you're proposing that something like the FC second level error
> recovery be performed, on failure of the abort (?).
>
See Julians reply.

> >
> > 7. Targets MUST support the command retry functionality. Don't
> think this
> > functionality provides much benefit in its current state.
> > Consider this case:
> > a. tape locate command is issued with a 10 second ULP timer
> > b. command is dropped at the target due to a digest error
> > c. having seen no status for 8 seconds (for example) the iSCSI initiator
> > decides to retry the command.
> >
> > What happens with the timer on the first command? If it is not
> canceled, and
> > if status is not received within 2 seconds, an abort for the
> command will be
> > issued by the ULP.
>
> Agreed. This is an issue in some cases, since not enough I/O time would
> be available for the retry. However, the selection of the initial
> command retry timeout value (when the initiator determines that the
> cmdsn has not been acknowledged for a long enough period and decides to
> retry the command) and the decision on whether to retry should be taken
> by the initiator driver, based on how much ULP time remains for that
> I/O.
>
> (Again, this logic is mostly O.S. driver specific.)
>
> >
> > dap: a mechanism to initiate error detection/recovery would be
> beneficial.
>
> Could you expand on this some more ?
>

I would think the command retry logic is iSCSI driver specific, not OS
driver specific.

A mechanism to determine whether or not the command arrived at the target
would be beneficail for the retry functionality to be useful.
The mechanism should be initiated early enough in the ULP timeout window to
allow the iSCSI retried command the opportunity to complete.

Some options:

A. If the initiator issues a NOP-Out(immed=1), the target can send back the
expected CmdSN.

B. The target, upon a header digest error, sent back a reject or NOP-In with
the expected CmdSN. This would provide an indication to the initiator that
something happened and trigger the command retry, still hopefully within the
ULP timeout to allow for sucessful command completion.

C. Texting stating that an iSCSI initiator should (only) perform a command
retry when sufficient time remains for the command to completed. But, this
leads one down the path of device type specific behavior.

D. Remove the command retry functionality. I have yet to see it actually
being used. The use of command retry is really only applicable when header
digests are being used. Given TCP, probability of header digest usage, and
existing ULP tools for error detection and recovery, my preference is to
remove it. This should also remove clause 8.3 from the iSCSI spec:)


> >
> > 8. CRN Processing and behavior: spec currently references SAM-2 for CRN.
> >
> > Per SAM-2:
> > Command Reference Number (CRN):
> > When this argument is used, all sequential commands of an I_T_L
> nexus shall
> > include a CRN argument that is incremented by one. The initial,
> wrap, and
> > reset CRN values shall be one. The CRN value zero shall be
> reserved for use
> > as defined by the SCSI protocol. It is not an error for the application
> > client to provide this argument when CRN is not supported by the SCSI
> > protocol or logical unit.
> >
> > More text specifying the behavior of CRN in the iSCSI realm is
> needed. In
> > addition, a method to determine if CRN is being used (or not)
> is missing.
>
> We went through this discussion several months ago in another ips
> thread. CRN really belongs to the SCSI ULP and any mechanism to check if
> the device server supports CRN should be in a SCSI ULP mode page (like
> the Control Mode Page).
>
> As far as iscsi is concerned, both the iscsi initiator and target
> implementations MUST support CRN, since it is defined in iscsi command
> pdu.
>
> Why is such a method required at the SCSI LLP layer ?
>

CRN is in a protocol specific LUN mode page because it is protocol specific
at this time
(e.g., it is not applicable to parallel SCSI, SRP currently does not support
it).

iSCSI has a field for the CRN, but provides no text regarding its use, other
than the reference to SAM-2.
The semantics for CRN are defined in FCP-2 clause 4.3. Most of the text
could possibly be rolled into SAM-3,
but some is still FCP specific, item d):

d) The initiator shall not transmit the same CRN again until delivery of the
first FCP_CMND IU transmitted
with that CRN has been confirmed by receipt of an FCP_XFER_RDY IU, an
FCP_DATA IU, an
FCP_RSP IU, an ACK, or a response to an REC.

But, I could probably generalize the above text.

My concern is with existing FCP-2 compliant implementations that are
bridged/gatwayed into the iSCSI realm.

Dave

>
> --
> ##################################
> Santosh Rao
> Software Design Engineer,
> HP-UX iSCSI Driver Team,
> Hewlett Packard, Cupertino.
> email : santoshr@cup.hp.com
> Phone : 408-447-3751
> ##################################