Re: iSCSI: ISID kicker issue

["Jim Hafner" <hafner@almaden.ibm.com>]

John,

Some comments in line

Jim Hafner


John Hufferd/San Jose/IBM@IBMUS@ece.cmu.edu on 09/28/2001 05:48:59 pm

Sent by:  owner-ips@ece.cmu.edu


To:   Jim Hafner/Almaden/IBM@IBMUS
cc:   ips@ece.cmu.edu
Subject:  Re: iSCSI: ISID kicker issue




Jim,
You and David have gotten very esoteric on this topic.  So let me step back
a moment and see if I understood the kicker.

I had thought that the SCSI folks were trying to come up with a way to say
"If I reserve something to system x, it should not matter how many
different session that system has with me, the reservations will apply to
all."  This tends to mean that the problems of managing such things in a
wedge driver became some what easier.
<JLH>
First, the T10 folks are trying to come up with "additional and optional
features" that would move in the diretion you're saying. That is partly to
make the wedge-driver's job easier.  But that doesn't mean that iSCSI can
ignore the current defined and well-understood model.  We have to live with
the current model AND see how what we do might affect (enable or prevent
enabling) the new features.

Having the target provide the SSID, in my mind, will prevent the new
features from being made available.
</JLH>

But you folks then spun down a path of having the application define what
session it wanted to use, .....
<JLH>
Well, I didn't go down this path.  This is what I think David is suggesting
and I'm questioning.  In my mind, the lower layers (based on some policy)
establishes sessions well before the application kicks in.  The application
takes what it gets in whatever form the lower layer want to present it
internally to the application.  David seems to be suggesting that the
application drive session establishment.
</JLH>

I have never seen an application like that.  Why are we spending time on
that issue?  All we want to do is have the persistence  apply to the total
system instead of a specific port.
<JLH>
I haven't seen an application like this either, but maybe David has or is
expecting to see them.

I'm not sure all the T10 people would agree with your last statement.
There is resistence to the new proposals at T10.  In particular, whatever
comes out of T10 will NOT break (or even confuse) any existing
implementations of the current model.  That we do have to live with.

Keep in mind that there are two independent proposals in T10.  One would
"disregard the target port" (and this is moving forward slowly).  The other
would "disregard the initiator port" (this has been stalled for a while).
The latter is closer to what your "apply to the total system" words say,
but I think your intent is actually the join of the two.

My point with the "kicker" is that the suggested change to SSID generation
might completely disable iSCSI from using these new features (because we
won't have 'named' SCSI initiator ports anymore).
</JLH>

If that is the case then the Nexus is in effect bound to the iSCSI
Initiator Node Name only, and not to the iSCSI Initiator Node Name + ISID.
<JLH>
But then you can't have more than one session per initiator node to target
portal group!  (That would be two parallel nexus!).  Your suggestion is to
have all iSCSI initiators modeled as single SCSI Initiator Port'ed.

There are conflicting interests here (multiple connections per session,
unlimited numbers of sessions, limitations of the SAM-2 and SPC-2 models,
etc.).  When you try to open one up twist on the model for simplification,
it tends to trample on another.

As I've said, I've been around this whole topic from every angle I can
think of, and where I ended up (with the model as defined in draft -08) was
the only option I found that provided the most flexibility for all issues,
properties, models,...

I haven't seen anything that would cause me to change my opinion about the
right model.
The issue in my mind is still (a) what action does the target perform to
enforce the ISID RULE (this is what started this tangled web of threads)
and (b) what is the best way to implement the initiator so that it doesn't
unknowingly break the ISID RULE and so cause the target to implement its
'action'.
</JLH>

I think this should be straight forward to implement.  I do not see the
problem with the Kicker.
<JLH>
The problem is two fold (as I've said above).  Either, you don't name SCSI
initiator ports and so make it very difficult for the new features (and
even for the current model) OR you have only one (named) SCSI initiator
port and end up limiting the session construction flexibility that one
might want.
</JLH>

However, you and David have been working so hard at this, I am afraid that
I have missed something.  Please lets try it again.  How far off am I?

<JLH>
One of my points is that I think it is important that iSCSI provide Names
for SCSI Initiator Ports.  I think these names have to come from the
initiator (not be generated by the targets).  The names have to provide
enough flexibility to allow maximum session multiplicity and be the basis
for SCSI Target identification of the SCSI Initiator Port for the purposes
of (most importantly, but not exclusively) SCSI reservations (of today and
tomorrow).   The only thing I've found so far is ISID on which to hang that
property.  Other proposals all seem to end up (to me) as either having less
functionality (e.g., more session restrictions) or duplicate functionality.
</JLH>
.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: hufferd@us.ibm.com


Jim Hafner/Almaden/IBM@IBMUS@ece.cmu.edu on 09/28/2001 04:51:58 PM

Sent by:  owner-ips@ece.cmu.edu


To:   John Hufferd/San Jose/IBM@IBMUS, ips@ece.cmu.edu
cc:
Subject:  Re: iSCSI: ISID issue




John,

On the surface, what you're proposing (having the target name the SCSI
Initiator Port) at least gives the port an identity.  But I wouldn't call
it a name (persistent, immutable, etc).  Besides, the target has no vested
interest in reusing the ISID (and some resource issue in not).  The
initiator on the other hand may have a lot of reasons to "show the same
face" for its side.   The "kicker" is one of those reasons.  The target can
always use different ISIDs and never have to worry about support for the
"kicker".

Jim Hafner


John Hufferd/San Jose/IBM@IBMUS@ece.cmu.edu on 09/28/2001 03:39:24 pm

Sent by:  owner-ips@ece.cmu.edu


To:   Jim Hafner/Almaden/IBM@IBMUS
cc:   ips@ece.cmu.edu
Subject:  Re: iSCSI: ISID issue




Jim,
You might consider the thing that passes out the ISID  to be on the Target
side.  You folks all yelled at me about my manic desire to have the target
assign both the ISID and the TSID, and saw no  reason why we should not
just call it SSID or SID.  But let me try it again, and see if it helps
this model.  Suppose, in addition to the Target setting the TSID as it does
today, it also assigns the ISID.

Now since the Session is not established until we go into Full Featured
Mode (before that it is only a connection), all you have really done is
move the routine that you described as assigning the ISID,  to a remote
location.  The Full Initiator session name (made up of the iSCSI Node Name
and ISID) is technically established before there is a session.  So it
should be the same difference as far as your model goes, but you would have
to keep the ISID as a separate item not just as a Target assigned SID.

Does that seem right to you?

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136
Internet address: hufferd@us.ibm.com


Jim Hafner/Almaden/IBM@IBMUS@ece.cmu.edu on 09/28/2001 02:41:03 PM

Sent by:  owner-ips@ece.cmu.edu


To:   Black_David@emc.com, ips@ece.cmu.edu
cc:
Subject:  Re: iSCSI: ISID issue




David,

See my reply to your other note on the "kicker" thread.  But I have a
comment here too.

Regardless of how we go about this 'who assigns the SSID' question, we
still have a fundamental question about what to map the SCSI Initiator Port
to AND what its 'SCSI Initiator Port Name' should be.  WIthout an initiator
generated (or owned) name, we can't have such a beast (we can have SCSI
Initiator Ports, but no name can be associated with them).   That will
prevent iSCSI from being able to take advantage of things that SCSI has so
far and will in the future better enable because of persistent named
initiator ports.

The ISID was the most obvious thing to choose.
Take that away and the whole thing has to get stirred up again.

I sympathize with the issue of configuration.
However, if we can't solve the iSCSI Initiator Name (IIN) problem (and
avoid the FC Nodename debacle), then it's true that configuration of ISIDs
will also be a problem.
I was expecting that we can solve the IIN problem by defining at least a de
facto standard that could be put in place today.  I was hoping (perhaps
naively) that we could solve the ISID allocation problem at the same time.
The whole point of stating all this up front now was in hopes of 'learning
from the FC mistakes'.

Let me add that (from private discussions) I've come to realize that the
current wording is perhaps not exactly reflective of the real requirement
for ISIDs.

The current wording says "partition". Some have come to interpret that as a
static (once at boot, sort of) partition. That actually goes too far.  What
is really needed (from the model point of view) is a service in the iSCSI
Node that allocates ISIDs for use by session creators in the initiator
portal groups.  Whether that is implemented passively by static
configuration (static partition) or by dynamic partition (partitions can be
reconfigured on the fly) or by an api to allocate on demand (on-line
algorithm) - doesn't really matter.  The relevant point is that something
(at the node level) has to pass them out for use in session creation within
the portal groups *in such a manner so as not to break the ISID RULE*.  The
static partition was one implementation that I thought was doable today.

Perhaps such a service is untenable.  If so, then I don't know that we can
put any weight in the model to the notion of an iSCSI Initiator Node!  If I
can't have that simple function, there can't be any more complex functions
like failover, error recovery, etc..

In any case, there is still open (as far as I know) the issue of "what
happens if a parallel nexus is attempted" (regardless of the definition of
SCSI Initiator Port).

Jim Hafner


Black_David@emc.com@ece.cmu.edu on 09/28/2001 12:17:43 pm

Sent by:  owner-ips@ece.cmu.edu


To:   ips@ece.cmu.edu
cc:
Subject:  iSCSI: ISID issue



Attempting to pick this back up, and selectively quoting from
the (extensive) email on this topic, here's where I think we
are on this.  The issue is whether to eliminate ISIDs, so that
iSCSI sessions are identified solely by target-assigned TSIDs.

The underlying concern is reliability of coordination of ISIDs
among independent iSCSI implementations that make up a single
iSCSI Initiator (one iSCSI Initiator Name), as originally stated
by Nick Martin:

> In particular this enables independent agents within the same initiator
to
> attempt a login without knowing all ISIDs in use by other agents.  Each
> agent would know the ISID of sessions it had successfully established,
but
> not the ISIDs for sessions established by others.  It can use the ISIDs
it
> knows to recover sessions it owns.  If an agent gets a failure attempting
to
> establish a new session, it would pick a different ISID and retry (or
just
> quit), rather than disrupting a session of another agent.
>
> I have a big problem with A) where independent agents must know the ISIDs
> established by others in order to do no harm.

There have been a couple of attempts to address this by requiring
ISID coordination to be done via administrative or OS means, and
allowing arbitrary failures when it is not done correctly.
Unfortunately, these attempts have failed to distinguish between
incorrect/buggy iSCSI protocol implementations (which can cause
all sorts of problems and arguably deserve whatever bad results
they get) and administrative errors such as mis-entering a config
parameter or miscopying a parameter from a hand-written configuration
sheet (for which there is benefit in limiting the damage that results,
as we can expect such errors to occur no matter how carefully
iSCSI and its management tools are implemented).  The first
attempt was to mandate a management interface - Jim Hafner wrote:

> ... mandate a management interface for setting/configuring ISID
> partitions and the problem goes away of non-cooperating HBAs.

We can definitely mandate the existence of such an interface (actually
ISID configuration interfaces for each iSCSI implementation), but we
cannot mandate their correct use in all circumstances.  We could decide
that it's ok for minor mistakes in using that interface to result in
major damage, but that may not be the best design approach.

The second attempt was to strongly encourage automatic configuration
mechanisms in OS platforms - Jim Hafner wrote:

> The whole reason we put in the draft the "SHOULD partition" ISIDs among
> portal groups and why it is so prominent is to get all the people
building
> these components to agree NOW to the OS-specific mechanisms to achieve
it.
> First recognize the need and THEN to define the mechanism (and I've said
> that the mechanism isn't hard, we (as vendors, not necessarily within the
> specification) just have to agree on it).

Much as I believe iSCSI is important, I think this is essentially an
exercise in wishful thinking - the "SHOULD partition" warning seems akin
to firing a BB pellet across the bow of an aircraft carrier - it will
likely be ignored.  I don't think iSCSI is in a position to drive this
sort of change into existing OS device driver infrastructures - rather
I think it's incumbent upon us to make sure that it can exist cleanly
within them.  Jim goes on to say:

> We're trying to prevent exactly the problem David (I think) mentioned
with
> FW Nodenames never taking on the role they should have.  We're posting
> right up front an implementation (strong) recommendation to enable both
> assignment of Initiator Name (from outside the HW or SW) and of ISIDs
(from
> outside the HW or SW).   This enables the protocol to function at its
best.
> If people don't want to implement to this recommendation, then they'll
pay
> the price with either  inter-vendor interoperability problems (not with
the
> wire but within a given initiator) or with much more complex management
> issues (a la FC Portnames).

And I'm concerned that having failed to learn from the Fibre Channel
history, we may be condemned to repeat it.  The cross-HBA interfaces to
coordinate the Node WWN never came into existence despite the best
intentions
and efforts of those involved in Fibre Channel, and they would have been
no more complex than the ISID coordination (e.g., find all the possible
Node WWNs, pick the numerically smallest).

An issue has been raised about why the Target is better suited to assign
session IDs than the Initiator.  I've seen at least two good answers to
this - Eddy Quicksall points out that this is fundamentally about managing
Target-controlled resources:

> Now, I'm wondering why we are even trying to use the ISID to reset a
session
> when we should be using the TSID ... because the target can produce
unique
> TSIDs and use that to identify the session much more easily than using
the
> combination of InitiatorName and ISID.

And Sandeep Joshi points out that Targets tend to have a single entity
controlling their entire implementation, unlike Initiators:

> ..the target may not be monolithic
> but one assumes it would atleast be "monogenic" (single-vendor)
> thereby enabling it to disallow multiple nexuses being
> started with the same <initiatorName,ISID>
>
> The monogenic property may not hold for initiators so
> a scheme which works without HBA cooperation is preferred
> over one which requires cooperation.

I think Jim Hafner's "kicker" issue of T10 changing reservations to be
associated only with SCSI Initiator Ports is a major problem for iSCSI
*independent* of whether ISIDs exist or not -- I don't think keeping ISIDs
in their current form is sufficient to address that issue and may in fact
be the wrong way to go about it, as I'll explain in a separate message.
I now believe this issue to be orthogonal to whether ISIDs remain, but
folks will have to read that separate message to see whether they agree.

So, after reviewing all the email on this, I definitely don't see
consensus on whether to keep ISIDs, but I'm seriously concerned
that we are repeating the mistake Fibre Channel made with Node Names
and will suffer the resulting consequences - iSCSI Initiator Names
will get bound to HBAs rather than OS images in order to make absolutely
positively sure that ISID conflicts cannot happen.

At the risk of starting yet another long discussion, comments?
--David

--------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140     FAX: +1 (508) 497-8500
black_david@emc.com       Mobile: +1 (978) 394-7754
---------------------------------------------------