Re: iSCSI: proposal to solve the ISID issues

["Mallikarjun C." <cbm@rose.hp.com>]

Jim,

I am in broad agreement with you on your suggestion,
and I agree that the ISID-configurability is a derivative 
requirement - but my argument is that it's nevertheless 
a crucial one to warrant making it explicit, but I can 
live with the proposed compromise of making it a NOTE.

Some comments -

>There is a requirement
> that vendor NICs (managed by the same vendor driver) be configurable in
> their ISIDs.

I am kind of perplexed on this underlying assumption that 
a vendor NICs are "managed by the same vendor driver".  As
you know, this is not always true.  IMHO, regardless of 
vendor and driver affiliations, an iSCSI initiator NIC MUST 
have external ISID-configurability - that's a design constraint
on a NIC vendor (from the ISID rule)! [ Note that I am NOT 
suggesting that the API be "public", it's upto the NIC vendor 
on allowing third-party drivers - and I believe most would. ]

With that, I would suggest the following wordsmithing, but
I will not press it any further... 

  NOTE: For correct behavior (in particular with respect to the ISID
rule), a
  vendor of iSCSI hardware implementations (e.g., NICs or HBAs) must
allow
  for configuration and coordination of ISIDs for all sessions managed
by
  multiple instances of that hardware within a given iSCSI Node.  Such
  configuration might be either static (e.g., partitioned across all the
NICs
  at initialization) or dynamic (e.g., on-line allocator).

I hope the suggested wording (or something close to it) on the 
Node name would make its way in.

Finally, thanks for so patiently driving this issue!
-- 
Mallikarjun 


Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
cbm@rose.hp.com


Jim Hafner wrote:
> 
> John and Mallikarjun,
> 
> I appreciate where Mallikarjun wants to go here.  There is a requirement
> that vendor NICs (managed by the same vendor driver) be configurable in
> their ISIDs.  However, that requirement is more a derivative conclusion of
> the rules (for a good and correct implementation) than it is a protocol
> requirement.  A vendor that doesn't do this only steps on his own toes when
> more than one of his cards is in a system!
> 
> Perhaps we compromise and make this a big NOTE:
> 
> NOTE: For correct behavior (in particular with respect to the ISID rule), a
> vendor of iSCSI hardware implementations (e.g., NICs or HBAs) must provide
> either driver software or external and public APIs to allow for
> configuration and coordination of ISIDs for all sessions managed by
> multiple instances of that hardware within a given iSCSI Node.  Such
> configuration might be either static (e.g., partitioned across all the NICs
> at initialization) or dynamic (e.g., on-line allocator).
> 
> We can make the iSCSI Node Name a stronger requirement (that is, outside of
> a NOTE).
> 
> Does that help?
> 
> Jim Hafner
> 
> John Hufferd/San Jose/IBM@IBMUS@ece.cmu.edu on 10/29/2001 06:47:31 pm
> 
> Sent by:  owner-ips@ece.cmu.edu
> 
> To:   cbm@rose.hp.com
> cc:   ips@ece.cmu.edu
> Subject:  Re: iSCSI: proposal to solve the ISID issues
> 
> I have no problem with the Must have a way to have the Node name set.
> However, I do not grasp the problem you state here:
> 
> "I don't believe the type of the ISID matters, nor does
> the NIC capability (coordinating vs not).  If two initiator
> NICs share the same node name, they better not re-use the
> same ISID to a given target portal group.  I was trying
> to capture the consequences of this reality."
> 
> If the ISID includes the Vendor ID, I do not see the problem, unless the
> vendor can not assign the rest of the ISID.  And I would say that it is the
> vendors problem to work out.  If you are saying that with NICs (not HBAs)
> that the SW Drivers will have a problem coming up with the ISID, I do not
> understand that either since most software is built by a Vendor, and if not
> the Random type should work.
> 
> Now in case you are saying something about the creation of a second Session
> from the Initiator, which is to access the same Target Node.  Then by
> definition, the ISID must be unique. (and it will probably need a Wedge
> Driver to operate correctly).  I would think that a vendor should be able
> to come up with a unique qualifier for the session since they have 64K
> numbers to chose from.  Remember the vendor only needs to coordinate the
> lower 2 bytes of the new ISID, with itself.  So if they can not do that,
> ... Oh well ...
> 
> Are we on the same page yet?
> 
> .
> .
> .
> John L. Hufferd
> Senior Technical Staff Member (STSM)
> IBM/SSG San Jose Ca
> Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> Home Office (408) 997-6136
> Internet address: hufferd@us.ibm.com
> 
> "Mallikarjun C." <cbm@rose.hp.com>@core.rose.hp.com on 10/29/2001 06:35:55
> PM
> 
> Please respond to cbm@rose.hp.com
> 
> Sent by:  cbm@core.rose.hp.com
> 
> To:   John Hufferd/San Jose/IBM@IBMUS
> cc:   ips@ece.cmu.edu
> Subject:  Re: iSCSI: proposal to solve the ISID issues
> 
> John,
> 
> I take it that you don't have an issue with the
> Node Name part of the wording.
> 
> I thought I answered your question by saying -
> 
> "My point though is that by banning a duplicate nexus (that an
> inadvertently re-used ISID allows), we are  making the ISID
> (dynamic/static) distribution an imminent hard requirement
> for all NICs in a given iSCSI initiator Node"
> 
> The unspoken assumption above is that we would want to
> allow multiple sessions between an initiator node and
> a target portal group.
> 
> I don't believe the type of the ISID matters, nor does
> the NIC capability (coordinating vs not).  If two initiator
> NICs share the same node name, they better not re-use the
> same ISID to a given target portal group.  I was trying
> to capture the consequences of this reality.
> 
> Regards.
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> MS 5668   Hewlett-Packard, Roseville.
> cbm@rose.hp.com
> 
> John Hufferd wrote:
> >
> > Mallikarjun,
> > I do not understand the purpose for your proposed wordage
> >
> > "All iSCSIinitiator hardware implementations MUST in addition also
> support
> > the operational ISID values to be (either statically or dynamically)
> > externally configurable."
> >
> > Why is this so important that it is a MUST.  The purpose of the proposal
> is
> > to eliminate the need for the above.  At most it should only apply to the
> > vendors (Probably Software and at Universities and IT shops) that use the
> > RANDOM ISID type code.
> >
> > I do not see the need elsewhere.  At most it could be "MAY".
> >
> > .
> > .
> > .
> > John L. Hufferd
> > Senior Technical Staff Member (STSM)
> > IBM/SSG San Jose Ca
> > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> > Home Office (408) 997-6136
> > Internet address: hufferd@us.ibm.com
> >
> > "Mallikarjun C." <cbm@rose.hp.com>@ece.cmu.edu on 10/29/2001 05:24:20 PM
> >
> > Please respond to cbm@rose.hp.com
> >
> > Sent by:  owner-ips@ece.cmu.edu
> >
> > To:   Jim Hafner/Almaden/IBM@IBMUS
> > cc:   ips@ece.cmu.edu
> > Subject:  Re: iSCSI: proposal to solve the ISID issues
> >
> > Jim,
> >
> > Thanks for the quick response.
> >
> > As much as I would like to avoid consuming more bandwidth
> > on this topic, let me make one last strawman for wording below.
> >
> > > What words do you suggest?
> > ...
> > > And a lot depends on how
> > > functional the NICs are: (a) just network nics, (b) protocol nics, but
> > not
> > > session coordinating nics (c) full session coordination nics (with
> driver
> > > assist) (d) fully autonomous session nics.   How do we make a
> requirement
> > > that fits all the cases correctly?
> >
> > I have been implying iSCSI NICs in all my postings so far -
> > sorry, may be that wasn't very clear.  "iSCSI Node name" comment
> > obviously wasn't targeted at simple network NICs.  So clearly (a)
> > is out of discussion.  But you are right - I wasn't very precise.
> > Here's a strawman -
> >
> >    All iSCSI hardware implementations (as in NICs) MUST allow
> >    the iSCSI Node Name to be externally configurable.  All iSCSI
> >    initiator hardware implementations MUST in addition also
> >    support the the operational ISID values to be (either statically
> >    or dynamically) externally configurable.
> >
> > > Making them configurable (at
> > > initialization time, e.g., as a range of values) or making them dynamic
> > > (the driver generates them on demand) both fit the mold.  So, I don't
> > think
> > > this one is a hard requirement.
> >
> > I agree that my earlier "range" wording was too constraining.
> > My point though is that by banning a duplicate nexus (that an
> > inadvertently re-used ISID allows), we are  making the ISID
> > (dynamic/static) distribution an imminent hard requirement
> > for all NICs in a given iSCSI initiator Node - except it is
> > not sufficiently explicit.  Please consider the above proposed
> > wording.
> >
> > > Well, I don't think so.  If each vendor implements "conservative reuse"
> > > within their own ISID namespace on their own NICs, then you get this
> > > property system wide.
> >
> > You are right, I guess I briefly overlooked the fact that ISID
> > includes the vendor identifier within, per the new proposal.
> >
> > I am personally okay with making "conservative reuse" a
> > SHOULD.
> >
> > Regards.
> > --
> > Mallikarjun
> >
> > Mallikarjun Chadalapaka
> > Networked Storage Architecture
> > Network Storage Solutions Organization
> > MS 5668   Hewlett-Packard, Roseville.
> > cbm@rose.hp.com
> >
> > Jim Hafner wrote:
> > >
> > > Mallikarjun,
> > >
> > > Comments in line below
> > >
> > > Jim Hafner
> > >
> > > "Mallikarjun C." <cbm@rose.hp.com>@core.rose.hp.com on 10/29/2001
> > 12:17:49
> > > pm
> > >
> > > Please respond to cbm@rose.hp.com
> > >
> > > Sent by:  cbm@core.rose.hp.com
> > >
> > > To:   Jim Hafner/Almaden/IBM@IBMUS
> > > cc:   ips@ece.cmu.edu
> > > Subject:  Re: iSCSI: proposal to solve the ISID issues
> > >
> > > Jim,
> > >
> > > Thanks for the excellent summary.  Some questions/comments -
> > >
> > > - I assume that the NICs must enable the configuration of
> > >   iSCSI Node name even in this proposal.  If it is so, please
> > >   call this out as a hard requirement in the main modelling
> > >   discussion.
> > >
> > > <JLH>
> > > This requirement doesn't change from before (but how it gets written
> into
> > > the spec may differ -- and we've had this discussion before).  If a
> > vendor
> > > doesn't allow configuration of his NIC's iSCSI Node name, then his NICs
> > > will be acting as their own iSCSI Node (that is, not representing the
> > > system they live in).  One can argue that this is a legitimate
> > > implementation (just as writing user-level code that uses its own iSCSI
> > > Node name).  So a "hard requirement" may be a bit too strong.  However,
> I
> > > will go with the will of the group on this point.
> > >
> > > What words do you suggest?
> > > </JLH>
> > > - In the case of multiple NICs from the same vendor operating
> > >   in an end node, it appears to me that the proposal implicitly
> > >   assumes an ISID-range to be configurable among those NICs
> > >   (perhaps in vendor-unique ways, which is always what I expected).
> > >   If this is a correct inference, there is again a hard
> > >   requirement on the NICs to make the ISID range configurable.
> > >   Please comment on any subtle qualitative change from the
> > >   earlier situation that I may be missing.
> > >
> > > <JLH>
> > > Well, the point is that the vendor can manage his own namespace for his
> > > NICs anyway he/she/it wants to.  Making them configurable (at
> > > initialization time, e.g., as a range of values) or making them dynamic
> > > (the driver generates them on demand) both fit the mold.  So, I don't
> > think
> > > this one is a hard requirement (though that is probably how most
> vendors
> > > will implement their NICs).  In effect, the proposal gives vendors more
> > > flexibility in their own space, without causing heterogenous
> > > interoperability problems within the host.  And a lot depends on how
> > > functional the NICs are: (a) just network nics, (b) protocol nics, but
> > not
> > > session coordinating nics (c) full session coordination nics (with
> driver
> > > assist) (d) fully autonomous session nics.   How do we make a
> requirement
> > > that fits all the cases correctly?  You clearly have in mind a specific
> > > level of implementation within the NICs, but that may not be
> everybody's
> > > model.
> > > </JLH>
> > >
> > > > (2-e) Its debatable whether "conservative reuse" is a MUST or a
> > > > SHOULD.  My personal opinion is "SHOULD", because many systems,
> > > > particularly low-end that don't use reservations, can function more
> or
> > > > less OK without it.
> > >
> > > It seems we're attempting to set ourselves up for future in
> > > discussing the above requirement.  Some questions -
> > >         - It appears to me that the "conservative reuse" can not
> > >           be enforced for initiators hosting NICs from different
> > >           vendors (since the proposal allows ISID namespaces to
> > >           be totally non-overlapping between non-homogenous NICs).
> > >           Is this a fair assessment?
> > > <JLH>
> > > Well, I don't think so.  If each vendor implements "conservative reuse"
> > > within their own ISID namespace on their own NICs, then you get this
> > > property system wide.  As before, by owning their own piece of the ISID
> > > namespace, they can implement what they want.  So, you may have a
> > situation
> > > where some of your installed nics have this property and some don't.
> > > You'll find out (if your environment really needs conservative reuse)
> > that
> > > you haven't got it, and it becomes a marketing/purchase spec
> requirement.
> > > </JLH>
> > >      - Do you see a particular disadvantage for low-end systems
> > >           if it's mandated (aside from the fact that they may be able
> > >           to live without it)?
> > > <JLH>
> > > No, but I don't see any way to really enforce it (or even really test
> > it).
> > > It's not a requirement of the protocol, per se.
> > > </JLH>
> > >      - Do you see any corner cases not being met (for future)
> > >           if we don't make it a MUST (since you said "more or less
> OK")?
> > > <JLH>
> > > No, I can't see that far into the future! :-{).  One reason I'm being
> > cagey
> > > here is that I'm finding it difficult to find the right words to
> specify
> > > this as a hard requirement (but I'm no technical writer either!).
> > > </JLH>
> > >
> > > Regards.
> > > --
> > > Mallikarjun
> > >
> > > Mallikarjun Chadalapaka
> > > Networked Storage Architecture
> > > Network Storage Solutions Organization
> > > MS 5668   Hewlett-Packard, Roseville.
> > > cbm@rose.hp.com
> > >
> > > Jim Hafner wrote:
> > > >
> > > > Folks, (David Black in particular).
> > > >
> > > > Apologies for the long note, but the issue is complicated.
> > > >
> > > > The NDTeam have a proposal to resolve the concerns surrounding use of
> > > > ISIDs as essential components (along with iSCSI Initiator Name) of
> > > > SCSI Initiator Portname.  (This was rooted in private discussions
> > > > between John Hufferd, Julian and myself -- and came about as a result
> > > > of the lengthy (and boring) discussions mostly myself and David
> > > > Black.)
> > > >
> > > > There are two somewhat orthogonal issues involved in this discussion:
> > > >
> > > > 1) How to coordinate ISIDs in the initiator to minimize the risk of
> > > > accidentally breaking the ISID RULE (no parallel nexus) when
> > > > independent vendors co-exist in the same OS.
> > > >
> > > > 2) What ISID "reuse" rules should be specified to facilitate current
> > > > and future (soon?) SCSI reservation semantics (and also internal OS
> > > > views of SCSI Initiator Ports).
> > > >
> > > > To address these two issues, we (NDT) propose the following:
> > > >
> > > > 1) Enlarge the ISID field in the Login Request and Response to 6bytes
> > > > and structure it with a component that corresponds to a "naming
> > > > authority" (essentially the vendor generating the ISID).  So vendors
> > > > each have a piece of the ISID namespace to work with their own
> > > > components (HBAs, SW, etc). More details below.
> > > >
> > > > 2) ISIDs (within the namespace of a given iSCSIName) SHOULD be used
> as
> > > > conservatively as possible ("conservative reuse").  What this means
> is
> > > > that a given ISID should be reused for as many sessions (across
> > > > multiple target portal groups) as possible (but never to the same
> > > > target portal group twice -- that's the ISID RULE).
> > > >
> > > > NOTES and ADVANTAGES:
> > > >
> > > > (1-a) Each vendor owns his own piece of the ISID namespace, so in
> > > > effect, at the vendor-level, this provides a "partitioning" of that
> > > > namespace.
> > > >
> > > > (1-b) We don't need to specify an OS infrastructure requirement for
> > > > configuration of ISIDs -- each vendor can do it any way it chooses
> > > > (within its naming authority).
> > > >
> > > > (1-c) Breaking of the ISID RULE will only occur if a vendor messes up
> > > > its own implementation.
> > > >
> > > > (1-d) This is not fundamentally different from David Black's
> > > > suggestion for a "key"; it just (a) defines it precisely and (b)
> > > > embeds it in an already defined (but enlarged) field.
> > > >
> > > > (1-e) Looking towards the future, as common ISID coordination
> > > > mechanisms are implemented between vendors (say, SNIA banding
> together
> > > > to define a precise API), this "naming authority" can be elevated to
> > > > the coordinating entity or even the OS vendor.
> > > >
> > > > (1-f) ISIDs have no global uniqueness property.  They need only be
> > > > unique between a named iSCSI initiator and iSCSI target portal group.
> > > > That means that a vendor implementation can use the same algorithm to
> > > > generate ISIDs (under its naming authority) in every machine (OS).
> > > >
> > > > (2-a) If a SCSI target (or logical unit) is holding state (say
> > > > persistent reservation) for a SCSI Initiator Port (named by iSCSI
> Name
> > > > and ISID), and the associated nexus/session is dropped for some
> > > > reason, "reuse" by that initiator of that ISID will restore to the
> > > > resulting new session that state (with no other action on the part of
> > > > the upper SCSI layers).
> > > >
> > > > (2-b) "Reuse" of an ISID on different sessions to (necessarily)
> > > > different SCSI Target Ports (iSCSI Target portal groups), will enable
> > > > the SCSI target/logical unit to recognize a common SCSI Initiator
> Port
> > > > for those two paths.  This facilitates future changes to SCSI
> > > > reservations (at least).
> > > >
> > > > (2-c) An initiator driver implementated with "conservative reuse" can
> > > > present to the OS a stable and fairly static view of the SCSI
> > > > Initiator Ports (one for each ISID).  Current OS driver stacks
> > > > (including current wedge drivers) that are built on the presumption
> of
> > > > such a stable view, will not need modification to handle the more
> > > > dynamic nature of iSCSI's SCSI Initiator Port. [Note: the existence
> of
> > > > a SCSI Initiator Port presented to the OS does not *require* that
> this
> > > > port discover all possible targets; what the initiator builds
> sessions
> > > > to using a specific ISID will determine what is presented to the OS
> as
> > > > "devices" and LUs visible to that SCSI Initiator Port (could be none
> > > > if that ISID is never actually used!).]
> > > >
> > > > (2-d) (Related to (2-c)): Adding a new (pseudo-static) SCSI Initiator
> > > > Port is as simple as configuring another ISID!
> > > >
> > > > (2-e) Its debatable whether "conservative reuse" is a MUST or a
> > > > SHOULD.  My personal opinion is "SHOULD", because many systems,
> > > > particularly low-end that don't use reservations, can function more
> or
> > > > less OK without it.  This is an open question.
> > > >
> > > > DETAILS:
> > > >
> > > > 1) ISID Structure:
> > > > The 6byte ISID structure would be split into three parts:
> > > >    "type" field (1byte) -- identifies the format of the naming
> > > >                            authority field
> > > >    "naming authority" field (3bytes) -- identifies the vendor (or
> > > >                            group of vendors) generating this ISID
> > > >    "qualifier" (2bytes) -- the free-space for ISID uniqueness
> > > >
> > > > The type field takes on three defined values with all other values
> > > > reserved:
> > > >          Type    naming authority format
> > > >          00h     IEEE OUI
> > > >          01h     IANA Enterprise Number (EN)
> > > >          02h     "Random"
> > > >
> > > > The first types two provide a mechanism to uniquely (world wide)
> > > > identify the naming authority.  A vendor with one or more OUIs and
> > > > possibly also Enterprise number MUST use at least one of these
> numbers
> > > > when it generates ISIDs.
> > > >
> > > > The "Random" type is for the case where the ISID generator (SW or HW)
> > > > is provided by an entity that has no OUI or EN.  This includes, for
> > > > example,
> > > > -- a user-written program that builds sessions (and has access to the
> > > > system level iSCSI Name)
> > > > -- a university or other organization providing the component
> > > > -- a testing tool
> > > >
> > > > For the "Random" type, the naming authority field should be a random
> > > > or pseudo-random number. (See below on how this affects "conservative
> > > > reuse").
> > > >
> > > > [Note: the "type" field needn't be this big, but NDT felt that (a)
> > > > 2bits was insufficient for the future, (b) the "type" field should be
> > > > first, and (c) the "naming authority" field should be byte-aligned.]
> > > >
> > > > 2) Conservative Reuse
> > > >
> > > > The "conservative reuse" principle of this proposal just says that
> > > > SCSI Initiator Portnames should be viewed as static things (as much
> as
> > > > possible) and reused (when feasible) to give the most stable
> > > > presentation of SCSI Initiator Ports to both the OS above and to SCSU
> > > > targets across the wire.
> > > >
> > > > Whereas the current draft says "The ISID RULE does not preclude the
> > > > use of the same ISID to different Target portal groups", the
> > > > "conservative reuse" requirement would say that such reuse (using the
> > > > same ISID to many target portal groups) is a SHOULD (or is that
> > > > MUST?).  So, in one implementation, each iSCSI Initiator Portal group
> > > > would get configured with iSCSI Name, and a (small) set of ISIDs.
> The
> > > > portal group might take this set of ISIDs as an ordered list.  The
> > > > first session to a Target portal group would use the first ISID, the
> > > > second to the *same* target portal group would use the second in the
> > > > list, etc.  The number of ISIDs would then define a configuration
> > > > parameter that can be interpreted as the maximum number of sessions
> > > > that can be built to a given target portal group.
> > > >
> > > > To facilitate "conservative reuse", the "qualifier" field of a set of
> > > > ISIDs SHOULD be generated using either a repeatable algorithm
> > > > (non-random or pseudo-random but based on a fixed seed) or any
> > > > algorithm and stored in a persistent location (e.g., registry or /etc
> > > > file).
> > > >
> > > > For the "Random" type, "conservative reuse" may not be an issue (say,
> > > > user application that doesn't care about reservations, etc.).  When
> it
> > > > is, the "naming authority" field SHOULD also be generated by a
> > > > mechanism similar to that for the "qualifier" field as specified
> above
> > > > (e.g., defined in the SW at compilation time.)
> > > >
> > > > DOCUMENTATING CHANGES:
> > > >
> > > > The iscsi drafts would need the following minor changes to support
> > > > this proposal:
> > > >
> > > > NDT doc
> > > > (a) define the structure of the ISID field (as described above)
> > > > (b) add some notes about implementation in the presense of
> > > > "conservative reuse" (e.g., that ISID should be configured once, say
> > > > at install time, then reused on subsequent reboots, even for "Random"
> > > > type).
> > > >
> > > > iSCSI MAIN doc:
> > > > (a) move the CID field in Login Request to another reserved field.
> > > > (b) expand the ISID field in Login Request and Response to encompass
> > > > the previous 4 bytes.
> > > > (c) add a sentence where the ISID field is defined indicating that
> > > > this field is a structured value, showing the structure (as above)
> and
> > > > point to the NDT document.
> > > > (d) add some text to "Note to Implementors" on "conservative reuse".
> > > >
> > > > Comments?
> > > >
> > > > Jim Hafner
> 
> --
> Mallikarjun
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> MS 5668   Hewlett-Packard, Roseville.
> cbm@rose.hp.com