Re: iSCSI: "kicker" reservation problem

["Jim Hafner" <hafner@almaden.ibm.com>]

David,

First, thanks for bringing this issue back up.  It sort of fell by the
wayside for more important things....

You wrote:
  If the proposed language is adopted to allow reservations to be shared
  across target ports, but be bound to initiator ports, the identities
  of the initiator ports have to be visible to the "application" so that
  it can control SCSI behavior with respect to reservations.
But that must also be true today (in a more static world).  If an
application requests a reservation for a logical unit on one I_T_L nexus,
it pretty much needs to know what path that is, so that it knows where to
direct it's IOs.  So, it needs some handle, identity, whatever for that
nexus within the operating system.  In the current reservation model, the
application probably can't deal with a completely opaque handle here. It
will need to know all the I_T_L paths so that is can do the registration
across all the paths.  In other words, it needs some internal identity for
the I and some external identity for the T.  That's true today and won't
change with the new proposal.

If the proposed language is adopted, then it can take better advantage of
this information, by recognizing that it only needs to register when the
I's are different (not when either the I or T is different).

In either case, it needs some internal identity for the "I'.   So long as
the iSCSI layer presents a representation of the 'I' as consistent with the
model, there is no problem.  The application doesn't need to see the ISID,
it needs to see the internal identity of the 'I' as the same if the
underlying ISIDs for the I_T_L nexus is the same.

At least it some layer has a way to tell that the I's are the same or not.
If the target generated SSIDs entirely, there would never be two nexus with
the same I (and different T) and so the whole multipathing issue gets much
more complex.  The application would ALWAYS have to work through each nexus
(as it does today, but for different reasons).

The issue here comes, not from the application getting a handle on the
I_T_L nexus (and perhaps its internals). It's more a question of what layer
is going to generate all those paths (nexus) in the first place?  Right
now, every SCSI Initiator Port does a greedy job of building nexus to every
SCSI Target Port it can find on the bus (fabric).  With iSCSI, that model
changes dramatically.  Someone has to drive this nexus building in a more
constructive manner since initiator ports need to be created on the fly.
Once their built, the rest goes according to plan (as it does today).  But
building them at all may require more application intervention OR each
iSCSI Initiator Portal Group does it's most aggressive best (subject to
some configuration limit of how many connections per session and how many
sesssions per target portal group and .....).

You wrote..
  At best, T10 is probably looking towards
  Fibre Channel in which session endpoints are static hardware interfaces.
  The easiest way to get back in line with T10 would be to forbid iSCSI
  sessions from spanning network interfaces, and then use the IP address
  of the network interface as the SCSI Initiator Port Name, but that would
  be a significant change from the approach we've been pursuing to date.
I think that's an unfair statement about T10.  The person pushing this new
stuff is active in SRP and in iSCSI and is fully aware of the issues and
implications.  But on the other hand, T10 can only deal with things which
fit within its model space.  That includes "named Initiator Ports" (with
the name being persistent and immutable).  They don't have to be static or
hw, just named.
This whole thing would have been a lot easier if we never had multiple
connection sessions.  But that was a choice made early.  Now we see what we
can do with what we have.

You wrote:
  Alternatively, we could invent yet another software name for the SCSI
  Initiator Port that exists primarily to cope with this new notion of
  reservation scope,
But as I've argued before, any other name still has the same problems of
namespace, use, generation, coordination.  The ISID was a name already
there and suited the purposes.   If we go away from ISIDs for other
reasons, then we still need another name, and we still need a rule that
says that 'named' port can't login twice with the same target portal group
and .... we're back where we are now.

I've been around this bend many many times and always come back to the same
place.  Can I stop now? :-{)

Jim Hafner


Black_David@emc.com@ece.cmu.edu on 09/28/2001 12:18:13 pm

Sent by:  owner-ips@ece.cmu.edu


To:   ips@ece.cmu.edu
cc:
Subject:  iSCSI: "kicker" reservation problem



In the midst of the ISID discussion, Jim Hafner tossed in the following:

> Now comes a slight kicker!  There is proposed language for SCSI that
would
> allow a device server to associate a reservation with a SCSI initiator
port
> *independent of the SCSI target port*, so I could have a path from one
SCSI
> initiator port through each SCSI target port and the reservation state
> would be equivalent over all the paths (sort of I_*_L nexus).  This new
> language is based on one fundamental principle, that is, that the SCSI
> initiator port has a world wide unique persistent intrinsic NAME on which
> to base the unambiguous recognition of that SCSI Initiator Port
independent
> of the target port.  In other words, with a name, the device server can
> recognize the same initiator port through any of its target ports and so
> grant equivalent access rights.

And then pointed out that leaving the ISID as currently specified would
be an easy way to accommodate this functionality.

After further thought, I agree that this is easy, but believe that it is
the wrong design approach because it'll make the current ISID allocation
problems significantly worse.  The reasoning starts by observing that
while SCSI makes reservations, it doesn't "own" them in that it has no
idea what the intended purpose of the reservation is - that knowledge
exists only in some "application" (e.g., backup, cluster) above SCSI.

If the proposed language is adopted to allow reservations to be shared
across target ports, but be bound to initiator ports, the identities
of the initiator ports have to be visible to the "application" so that
it can control SCSI behavior with respect to reservations.  For iSCSI,
I think this leads to applications needing to be involved in ISID
selection.
Consider an application that opens a new iSCSI session to a different SCSI
port on an iSCSI Target to which a session is already open.  How does
iSCSI know whether to reuse the same ISID as the existing session (so
that reservations can span the sessions) or use a new ISID (so they don't)?
The answer is that iSCSI does not have the information required to answer
that question - it has to be passed down from the "application" which is
the only entity that knows what the intended result is.  The upshot is that
the ISID coordination problem has just expanded in scope from multiple
iSCSI implementations (e.g., HBAs) to also include multiple "applications"
above SCSI - this is not a good thing.  For the sake of simplicity, I think
that if ISID remains, it should not be used for this sort of reservation
scope control.

In practice, I think
the problem is more fundamental -- the proposed T10 language appears to
be at odds with iSCSI's notion of dynamically created session endpoints
that span hardware interfaces.  At best, T10 is probably looking towards
Fibre Channel in which session endpoints are static hardware interfaces.
The easiest way to get back in line with T10 would be to forbid iSCSI
sessions from spanning network interfaces, and then use the IP address
of the network interface as the SCSI Initiator Port Name, but that would
be a significant change from the approach we've been pursuing to date.
Alternatively, we could invent yet another software name for the SCSI
Initiator Port that exists primarily to cope with this new notion of
reservation scope, but I tend to agree with Jim Hafner's concern that
this makes the mapping of SAM-2 SCSI Port concepts to iSCSI less
believable/tenable/etc.

There doesn't appear to be a good solution here.

Comments?
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------