RE: iSCSI:Target Centric ISID assignments

["Julian Satran" <Julian_Satran@il.ibm.com>]

Blowing away or not is again with us. A careless guy will set always the
blow-away bit.
You have just moved the management responsibility to the target under the
assumption that the target
is "monolithic". But, as you well know, this is not true for many targets.
You have to have an allocation scheme at
target and a garbage collection.

I suggest we give this to the ND team and let them debate for a while
before getting back to us.
We may want to have them look at all aspects of SSID allocation and use.

Julo



Black_David@emc.com@ece.cmu.edu on 08-09-2001 00:46:21

Please respond to Black_David@emc.com

Sent by:  owner-ips@ece.cmu.edu


To:   John Hufferd/San Jose/IBM@IBMUS, ips@ece.cmu.edu
cc:
Subject:  RE: iSCSI:Target Centric ISID assignments



John's description is actually over-complicated because:
- The ISID either goes away or becomes a Target portal group number.
- The session recovery mechanism that he portrays as added complexity
     already exists and isn't affected by presence or absence of
     the ISID.  It doesn't seem to be well-documented in -07.
Partitioning the TSID into Target Portal Group + ID within group
makes it easy to figure out where to go to recover or blow away
a session.  Whether or not the Target Portal Group is used doesn't
matter to the following description - the ISID vanishes from John's
description, making things simpler, viz.:

First Login:
1. Initiator contacts the target with TSID of zero.
2. Target assigns the SSID by filling in the TSID.
3. Initiator remembers the SSID=TSID somehow.
4. If a parallel Session is established (perhaps under a wedge driver) the
     same thing will happen -- Initiator sends TSID=0, and the
     Target will assign a different TSID, since it is keeping
     state on the TSIDs it has handed out to a specific iSCSI Initiator
Node.
     This will be a different NEXUS not associated with any other session
on
     that iSCSI Initiator Node.

Note that the target has the option of keeping TSID state globally, so that
TSID values are unique within a Target or a Target portal group.  This is
the Target implementer's choice, and can simplify session lookup.  This
particular comment applies independent of ISID removal, but is the reason
why I mentioned the idea of expanding the TSID, as I can foresee problems
with a 16 bit counter, but not with 24 or 32 bits.

5. If a Multiple Connection per Session was established, the  NON leading
     connection will have the TSID filled in, and things
     will continue as currently documented.
6. If the Initiator goes down, and the parallel sessions need to
     re-establish the NEXUS they supply the TSID.
6a. If they don't have the original TSID, they send TSID=0, and the Target
     will establish a new session.  The NEXUS that might have the
Persistent
     Reserves is still bound to the previous Session.
6b. If the New session wants to keep working with the old Persistent
     Reserves, it needs explicitly blow-away the Reserves and Reclaim
them (per
     normal) using the Persistent Reserves Command Set.
6c. If the TSID had been saved by the Initiator, and a logon is reissued,
     it can specify the TSID, and the Target should then know that
     a new session is starting.  The target should match the TSID with
     any outstanding session it has.  That is, the Target should either
     match it up the TSID with an old session (in some way), and
continue,
     or  it should Blow the OLD session away.  David suggested a "Blow it
     away Bit". (This sounds like option A and B all over again.)

Whether or not to have a "Blow it away Bit" vs. implicitly blowing away
the old session vs. failing the login is Option A/B/C all over again.
The X bit already has meaning in this context (for the 6c case the X
bit has to be zero in this case because 6d uses it), and hence can't be
reused here.

It's now considerably safer to blow away old connections because a non-zero
TSID only occurs on login when there was a previous session with that TSID
and some sort of error recovery or action on that previous session is
intended.
All the scenarios in which a "wrong" ISID causes a session to be blown away
incorrectly can't happen because the Initiator doesn't pick ISID values.

6d. If the ISID was saved but it took so long that the Target's session
     cleanup process, had thrown away the Old session, the Login just
continues
     as it does today.  The Initiator Session can determine if the
session
     continued, and it has it previous reserves, or if a new session was
     created without the Reserves.

Actual recovery and continuation of the session has to set the X bit.  If
X is set on a Login command, and there's no existing session to log into,
then the Login has to fail in some fashion - I'm not sure that's documented
at the moment, and I think it needs to be regardless of the course we take.

     So the Option here is to return an error message if there is no
     existing session.  The Initiator will then need to understand this
     and somehow cause the reserves to be issued again.  (Folks will say,
     that is what they do anyway so they will always do that, so don't
     worry about it.)

     (John's Comment:  This is now starting to get complicated again.)

Correct, but not relevant.  This set of complications exists regardless of
what we do about the ISID as they're inherent in the session case of Error
Recovery and the use of the X-bit on Login.  This is an existing session
recovery mechanism, not something that gets added as a consequence of ISID
removal.

> 7. In the case that the Initiator has "lost its way" and want to start
     again,  the initiator will Login with TSID=0.

This is exactly case 6a above.  John's case 7a can't happen.

> OK, after looking at the above.  It seems to me that we can have this
> assignment of ISID by the Target System. (It is kind of acting like a
third
> party ISID assigner.)  The rest of the documentation should be the same.

Well, actually, it's simpler to get rid of the ISID and just use the TSID.
For reasons that Jim Hafner can probably explain better than I can, using
the ISID field to hold the Target portal group number on login helps tidy
up some issues in that area (and it also makes it easy for an Initiator
trying to recover or destroy an existing session to figure out exactly
which target interfaces will know about that session vs. be clueless).

> The issue of Option A or B is still with us. However, the issue of the
> Rouge Initiator, that can not find its approprate ISID is solved. (Don't
> you just love the emotive way I put that.)

Keep in mind that the Rogue Initiator sure looks like a valid technical
objection to the Option A that seems to be otherwise favored in list
discussion.

> Also the assignment of ISIDs is made easier for some implementations.
> (However, the approach suggested by Jim Hafner seems simple enough.)

It's certainly simple logic.  The problem is that ISID assignment has to
be coordinated across all interfaces on a single Initiator - between code
from different vendors, administrative tools, and interaction with
logic in the OS platform that may be trying to coordinate ISID assignment,
there are many opportunities to get things wrong.

If this doesn't scare you, John's notion of having to coordinate ISID
assignment across all systems in a cluster because they share a common
iSCSI Initiator Name should.  FWIW, if the cluster example wants to
recover session state (e.g., persistent reserves) across host system
failures, it needs dynamic fault-tolerant ISID assignment across the
entire cluster (that should really scare you) or elimination of ISIDs
and ISID assignment (which is much simpler ;-) ).

Thanks,
--David

> -----Original Message-----
> From: John Hufferd [mailto:hufferd@us.ibm.com]
> Sent: Friday, September 07, 2001 4:54 PM
> To: ips@ece.cmu.edu
> Subject:
>
>
> Let me see if I have everything together in the following
> regarding the
> TSID centric assignment of ISID.
>
> First Login:
> 1. Initiator contacts the target with TSID of zero, and ISID of zero
> 2. Target assigns the SSID by filling in the ISID and its own TSID.
> 3. Initiator remembers the ISID somehow.
> 4. If a parallel Session is establish (perhaps under a wedge
> driver) the
> same thing will happen -- Initiator sends TSID=0 and ISID=0,
> except the
> Target will assign a different ISID, since it is keeping
> state on the ISIDs
> it has handed out to a specific iSCSI Initiator Node.  This will be a
> different NEXUS not associated with any other session on that iSCSI
> Initiator Node.
> 5. If a Multiple Connection per Session was established, the
> NON leading
> connection will have the SSID (with TSID and ISID filled in),
> and things
> will continue as currently documented.
> 6. If the Initiator goes down, and the parallel sessions need to
> re-establish the NEXUS they come back with the TSID but may
> or may not have
> the approprate  ISID (if they did not have a way to save it).
> 6a. If no ISID was used in the Initiator Login, the Target will
> re-establish a new session.  The NEXUS that might have the Persistent
> Reserves is still bound to the previous Session.
> 6b. If the New session wants to keep working with the old Persistent
> Reserves, it needs explicitly blow-away the Reserves and
> Reclaim them (per
> normal) using the Persistent Reserves Command Set.
> 6c. If the ISID had been saved by the Initiator, and a logon
> is reissued,
> it can specify the ISID, and NO TSID, and the Target should
> then know that
> a new session is starting.  The target should match the ISID with any
> outstanding session it has.  That is, the Target should
> either match it up
> the ISID with an old session (in some way), and continue, or
> it should Blow
> the OLD session away.  David suggested a "Blow it away Bit".
> (This sounds
> like option A and B all over again.)
> 6d. If the ISID was saved but it took so long that the
> Target's session
> cleanup process, had thrown away the Old session, the Login
> just continues
> as it does today.  The Initiator Session can not really
> determine if the
> session continued, and it has it previous reserves, or if a
> new session was
> created without the Reserves.  (Potential hitch.)  So the
> Option here is to
> return an error message if there is no existing session.  The
> Initiator
> will then need to understand this and somehow cause the reserves to be
> issued again.  (Folks will say, that is what they do anyway
> so they will
> always do that, so don't worry about it.)
> (Comment:  This is now starting to get complicated again.)
> 7. In the case that the Initiator has "lost its way" and want to start
> again,  the initiator will Login with either the current
> ISID, and TSID =0
> or with both =0.
> 7a. Using the current ISID should cause the Initiator to
> handle it like 6c
> above.  (Again, it sounds like the A, and B options).
> 7b. If both ISID and TSID are zero, it looks like 6a above.
>
> OK, after looking at the above.  It seems to me that we can have this
> assignment of ISID by the Target System. (It is kind of
> acting like a third
> party ISID assigner.)  The rest of the documentation should
> be the same.
> The issue of Option A or B is still with us. However, the issue of the
> Rouge Initiator, that can not find its approprate ISID is
> solved. (Don't
> you just love the emotive way I put that.) Also the
> assignment of ISIDs is
> made easier for some implementations. (However, the approach
> suggested by
> Jim Hafner seems simple enough.)
>
> Now the discussion should be, did I miss something important,
> and is the
> change worth it
>
> .
> .
> .
> John L. Hufferd
> Senior Technical Staff Member (STSM)
> IBM/SSG San Jose Ca
> Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> Home Office (408) 997-6136
> Internet address: hufferd@us.ibm.com
>