Re: iSCSI: handling of persistent reserves during initiator reboot

["Jim Hafner" <hafner@almaden.ibm.com>]

Tom,

Here's my two cents. Apologies for being long-winded.  In short, I think I
agree with most of your sentiment.

1) In order to accommodate Persistent Reservations (and a few other SCSI
things), I think it is important to have some mechanism for an initiator to
reestablish some nexus state information through logout/login.  The current
thinking in N&DT is that the initiator should reuse it's old ISID and the
target should respond with the old TSID (thereby rebuilding the nexus).
Note, the initiator starts with TSID=0, as if it was a new session.  The
target that remembers the old ISID/TSID pair (can reuse the old TSID). Keep
in mind that that target is already remembering some state information (the
reservation) about the ISID/TSID (and Names) relationship in the nexus, so
this isn't a big deal.

2) One way to view this SCSI requirement is that the SCSI nexus gets
rebuilt.  But that doesn't imply that the iSCSI session itself recovers.
Consequently, I would think that the iSCSI is as "fresh" as it need be.
Note that the nexus itself need not go back to the complete previous state
it was in.   Certain information needs to be restored for the nexus, but
not everything (there will be lost tasks and that's OK).  I think this is
"restored" nexus, not a "continued" nexus, so the requirements are very
different.

3) There is a symmetrical problem here that was lost in the discussion
quoted.  Namely, persistent reserves are more precisely a requirement for
the target to maintain/restore through the target's reset events (like
power cycles).  It is less of a statement about initiator's logout/login.
FCP added something even more general by the requirement that persistent
reservation state for an initiator be restored after network address
reassignment.  (The initiator may have moved address but it still the same
initiator by name so keeps some state through these underlying transport
events.) That's the problem that needs to be addressed.  We can think of
session crashes (connect drops), and target recycles and host rebooting as
events that tear down the underlying transport; such events should have
minimal impact on SCSI stuff.

4) These "minimal impacts" may in fact be a lot (like aborting of all
current tasks and clearing session state).

5) I'm supportive of a section in the main document (perhaps an appendix or
annex, but I don't care) that does the following:
  (a) describes the mapping of SCSI terms to iSCSI terms, in particular,
the nexus, the initiator and target devices and ports and their names and
identifiers, et al.
  (b) defines the SCSI state of a nexus that must be restored when the
nexus gets rebuilt (this includes but is not exclusively persistent
reservations)
  (c) defines the procedures for how that nexus gets rebuilt and the side
effect that has on other state information at the SCSI and iSCSI levels

Well that was probably more than 2 cents (or maybe it was more like 4
lire).

Jim Hafner


Thomas McSweeney/Raleigh/IBM@IBMUS@ece.cmu.edu on 04-24-2001 07:50:36 AM

Sent by:  owner-ips@ece.cmu.edu


To:   ips@ece.cmu.edu
cc:   iscsi-mib@external.cisco.com
Subject:  iSCSI: handling of persistent reserves during initiator reboot



I'd like to move this discussion from the iSCSI MIB exploder to the IPS
exploder, since it has implications on the iSCSI spec.

John Hufferd appended a question asking whether the target would reset its
session counters under the following conditions:

>And if the Host was rebooted but comes back to the Target
>with the same iSCSI name and ISID, of a broken connection --
>which has outstanding persistent reserves -- then the
>Target must respond with the corresponding TSID of the
>interrupted Session.   This means that the Session has
>continued, and the connection has also been continued.

My response was:

I disagree.  Certainly, it is a new connection, so at the target the old
connection object would disappear and be replaced.  I think it is a new
session also - spec section 1.2.3 states that Login with a null TSID is for
a new session.  Whether or not the old session is cleaned up is still a
matter of debate...  Section 6.7.3 mentions "Login with an implied Logout",
which seems to apply to this case.  However, that depends on how the
"multiple sessions between one initiator and one target" issue is resolved.

If the initiator retained enough info about the I-T nexus despite the
reboot to send the correct non-null TSID on the (re)Login, then it should
have saved all its counters too.  The target does not reset its session
object counters due to within-session connection recovery.

John replied:

>I did not say that the initiators saved the TSID, just their own ISID. The
>point is that it is a SAM-2 requirement that if Persistent Reserves are
>outstanding, the Initiator should be able to clean them up or continue to
>work with them after a boot.  This will mean that the target will have to
>detect that  Persistent Reserves are outstanding, and keep approprate
>information around for the reconnection (with the InitiatorName=xxxxxxx,
>and ISID of zzzzzzz).  The Target will then need to respond --- when a new
>session is being created for InitiatorName=xxxxxxx, and ISID or zzzzzzzz
>--- with the TSID used with the broken connection.  In this way the
>initiator can clean up or continuation of the Persistent Reserves.

>Persistent Reserves are a bit different then any other State that is kept
>by the Target.

I believe the Login which the rebooted initiator sends has to be for a
"leading connection", so it can learn the value of negotiated iSCSI
parameters like MaxConnections and FMarker that are only exchanged during
initial session activation.  So to me it's a new session.  It shouldn't
matter to the underlying SCSI resources whether the TSID matches the
previous session or not.   I'm not sure how the target should handle the
outstanding persistent reserves - I assume they can be moved to the new
session.  Now it's time for the real experts to chime in...

I'd like to see a new section added to Chapter 6 of the spec to describe
the handling of outstanding persistent reserves, since it is a special
recovery case.

As for the SNMP objects, I think the counts will be reset (old session
instance disappears, new session instance initialized to zero).  This would
make the target's counts consistent with the initiator's.  We plan to
discuss the tracking of long-term (across multiple sessions) usage per
initiator in Nashua.

Tom McSweeney
iSCSI Development, Storage Systems Group, IBM
Email: rf42tpme@us.ibm.com
Phone: (USA) 919-254-5634  (tie line: 444-5634)
Fax:   (USA) 919-254-0391  (tie line: 444-0391)