Re: iSCSI: "conservative reuse" requirement

["Mallikarjun C." <cbm@rose.hp.com>]

I think we're getting close to wrapping this up.

>Once that's done, what's the value in permitting operating modes
> other than "conservative reuse"?

I thought we already discussed this - that of two initiator NICs
each establishing a session (acting as independent initiator ports)
to a different target portal group.

>Is anyone
> making significant changes to this "bus walker" boot paradigm out there?

Not that I know of.  But keep in mind that the nexii established 
for the boot discovery process may not have to be kept around
persistently by initiators (as a matter of fact, I know that HP-UX
does not).  There are also other cases - DLKM drivers, dynamically
discovered targets etc - which do not have nexii from boot time.
But again, I prefer to leave it there since this is a little orthogonal 
to the current issue.

> - SHOULD use "conservative reuse" in setting up sessions

I guess this is okay, the other alternative/complement is wording 
(proposed earlier in this thread) that focuses on where conservative 
reuse is a MUST.

> - SHOULD use "conservative reuse" in setting up sessions from the same
>         Initiator Portal Group (set of IP addresses) to different Target
>         Portal Groups

In the sense you used "initiator portal group" here, it ought to be
defined
(otherwise, simply use "initiator node" in your sentence) as "the set of 
initiator portals sharing the same ISID Type & ISID Naming Authority
values
and sharing one ISID Qualifier name space, and that allow a session to
span 
an arbitrary subset of the portals."

IOW, if two NICs A and B from the same vendor (so same Type and Naming 
Authority) are present in a host (so share the ISID Qualifier space,
perhaps 
with a s/w "ISID allocator" in the host or whatever), but can not span a 
session across them, then they are technically two independent
"initiator 
portal groups".

Thanks.
-- 
Mallikarjun 


Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
MS 5668	Hewlett-Packard, Roseville.
cbm@rose.hp.com


Black_David@emc.com wrote:
> 
> > Thanks for the message.  Let me comment on what I agree with first.
> >
> > >if "conservative reuse" does not
> > > happen (breaking shared persistent reservations), the reason that
> > > it does not happen MUST be externally configurable as opposed to
> > > being an unchangeable internal characteristic of the implementation.
> >
> > Agreed.  If I understand you correctly, you're concerned that NICs
> > that are hard-coded not to allow single initiator port abstraction (i.e.
> that
> > don't do conservative reuse) will break the new shared persistent
> reservations.
> > I agree that it currently is an issue, I propose that we have a NIC
> requirement
> > in the implementation notes (like the node name configurability
> requirement)
> > that says something like - Every iSCSI NIC MUST provide a means of
> > enabling a strict "conservative reuse" of ISIDs across different target
> portal
> > groups.
> 
> Good.  Once that's done, what's the value in permitting operating modes
> other than "conservative reuse"?  The risk of multiple operating modes is
> that there'll be:
> - The mode that works, and
> - The mode that complies with the standards.
> Having only one mode avoids this problem.  Don't laugh too loudly -
> versions of this situation are occurring in Fibre Channel switches
> today.
> 
> > With that said, here's the idea (repeated in several places)
> > that I have a problem with -
> > > every possible connection is
> > > attempted, and the reasons that not every connection is established
> > > are external (and configurable, modulo physical constraints).
> >
> > First off, I am not sure what you meant by "connection" here, and
> "presented"
> > in your previous message.  I took it that you meant "establishing a SCSI
> nexus".
> > If that's indeed so, the assertion that it's only the external factors
> that decide
> > nexii is not always true.  Consider the case of an FC initiator
> > port discovering target ports using an FC Name Server, and an iSCSI
> initiator
> > port discovering target ports using a "SendTargets" command.  In either
> case,
> > the initiator SCSI port may not really establish a nexus with a discovered
> > target SCSI port, unless the application (or
> > even the SCSI class driver) really wants to do an I/O (starting with an
> open()
> > system call in Unix systems).
> 
> I strongly disagree with that.  Virtually all commercial OS implementations
> of SCSI do aggressive establishments of the nexii as part of the boot
> sequence,
> as SCSI code tends to expect a "bus walker" to find devices in the boot
> sequence, and make sure they work.  Put a trace analyzer to work on a boot
> and watch all the TEST UNIT READY and similar commands flow by.  Is anyone
> making significant changes to this "bus walker" boot paradigm out there?
> 
> There's also a subtle issue to watch out for here in that this delays
> error discovery - if the shared persistent reservations are being
> used by cluster software, discovering that the alternate path nexus can't
> be established when trying to fail over to it is way too late.  For things
> like quorum volumes, I'm fairly sure that cluster software checks all the
> access paths at initialization time, but can someone familiar with cluster
> software comment on whether the alternate access paths needed for failover
> of application volumes are checked in advance, vs. relying on the OS to
> complain if the SCSI connection to the device didn't set up correctly?
> 
> > But in any case, it appears to be in the implementation territory.
> 
> That's something I can agree with - namely that when nexii are established
> is an orthogonal issue to the session identifiers used to establish
> them, although I would caution implementers that there's a lot of code
> out there that operates under the "bus walker" paradigm and hence wants
> to see nexii (sessions) established aggressively.  In a code stack that
> doesn't operate under the "bus walker" paradigm, "conservative reuse"
> is still applicable, as it need only constrain what ISIDs have to be
> used, and not when they are used.
> 
> Taking a whack at requirements, I think there are at least two of them:
> - MUST support "conservative reuse" without requiring any iSCSI sessions
>         to be terminated in order to set up the required sessions (i.e.,
>         one can take the ISID from an existing session and a target portal
>         group to which that ISID is not in use and set up an iSCSI session
>         with that ISID without having to terminate any existing session for
>         ISID usage reasons - this is tricky to state precisely when the
>         possibility of allowing sessions to be set up on the fly well after
>         booting is allowed).
> - SHOULD use "conservative reuse" in setting up sessions from the same
>         Initiator Portal Group (set of IP addresses) to different Target
>         Portal Groups (i.e., reuse an existing ISID in preference to
>         creating a new one).
> I presume the "mapping police" will let us know if Initiator Portal Group
> is not the right term for the second requirement ;-).
> 
> Thanks,
> --David
> 
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
> black_david@emc.com         Cell: +1 (978) 394-7754
> ---------------------------------------------------