Re: iSCSI: "conservative reuse" requirement

["Mallikarjun C." <cbm@rose.hp.com>]

David,

Thanks for the message.  Let me comment on what I agree with first.

>if "conservative reuse" does not
> happen (breaking shared persistent reservations), the reason that
> it does not happen MUST be externally configurable as opposed to
> being an unchangeable internal characteristic of the implementation.

Agreed.  If I understand you correctly, you're concerned that NICs
that are hard-coded not to allow single initiator port abstraction (i.e. that
don't do conservative reuse) will break the new shared persistent reservations.
I agree that it currently is an issue, I propose that we have a NIC requirement
in the implementation notes (like the node name configurability requirement)
that says something like - Every iSCSI NIC MUST provide a means of
enabling a strict "conservative reuse" of ISIDs across different target portal
groups.

In the case of OSVs doing this in software, one would have to assume that
they cater to the app requirements that are likely to be deployed on that O/S.
IMHO, my proposed formulation (bottom) addresses that adequately.

With that said, here's the idea (repeated in several places) that I have a problem
with -
> every possible connection is
> attempted, and the reasons that not every connection is established
> are external (and configurable, modulo physical constraints).

First off, I am not sure what you meant by "connection" here, and "presented"
in your previous message.  I took it that you meant "establishing a SCSI nexus".
If that's indeed so, the assertion that it's only the external factors that decide
nexii
is not always true.  Consider the case of an FC initiator port discovering target
ports
using an FC Name Server, and an iSCSI initiator port discovering target ports
using a "SendTargets" command.  In either case, the initiator SCSI port may not
really establish a nexus with a discovered target SCSI port, unless the application
(or
even the SCSI class driver) really wants to do an I/O (starting with an open()
system call in Unix systems).  But in any case, it appears to be in the
implementation
territory.

Regards.
--
Mallikarjun

Mallikarjun Chadalapaka
Networked Storage Architecture
Network Storage Solutions Organization
Hewlett-Packard MS 5668
Roseville CA 95747


----- Original Message -----
From: <Black_David@emc.com>
To: <cbm@rose.hp.com>
Cc: <ips@ece.cmu.edu>
Sent: Wednesday, January 02, 2002 3:45 PM
Subject: RE: iSCSI: "conservative reuse" requirement


> This topic is not exactly amenable to short summaries ...
>
> > I disagree with your assumption that an initiator must
> > "present" *all* its ports to every target port in a target
> > device.  Each distinct initiator port (signified by a unique
> > ISID) may want to talk to only one target port (i.e. one portal
> > group) - that's a typical usage of multi-HBA (i.e. multi-port)
> > hosts talking to multi-FC port targets today.  As far as I
> > can understand, a mandatory "conservative reuse" would
> > preclude this usage.  Please comment if your visualization
> > of conservative reuse is different.
>
> The concern I have is based on the source of the connectivity
> constraints.  To begin with, I disagree with the notion that
> a FC imitator port "may want to talk to only one target port";
> Fibre Channel does not work that way because the connectivity
> constraints are imposed externally to the ports.  For the
> case of an m-way HBA host talking to a n-way FC port target,
> "conservative reuse" is attempted by the HBAs (each port has
> exactly one WWN), and if less than the full m x n set of connections
> results, it is due to an external absence of connectivity -
> either each HBA port is connected to one target FC port, or
> the fabric(s) involved are zoned in a fashion that this is
> the resulting logical connectivity.  In both cases, an HBA
> following the rules connects to everything that it is allowed
> to connect to.
>
> I have no problem with external physical or logical (e.g., VLAN,
> or even iSNS) connectivity constraints preventing all possible
> connections between a multi-port initiator and a multi-port target.
> What I have a problem with is an implementation making an
> internal decision that it will *never*:
>
>       present a single SCSI port abstraction to multiple target
>       portal groups (thus multiple SCSI target ports) of an iSCSI
>       target node,
>
> no matter what its administrators and users may want.  IMHO,
> that's not the implementation's decision to make - it needs to
> be made externally by the administrator who configures the
> system. In particular, if an implementation is operating over
> a single IP interface, there has been discussion on this list
> of using a different ISID for every target portal group that
> it connects to - shared persistent reservations will never work
> right if that is done, and there is nothing that the administrator
> can do in that situation to get them to work ... I think this needs
> to be prohibited if we're serious about following T10's direction
> on persistent shared reservations, unless T10 is going to
> provide a way to query the initiator to discover that it does
> not support shared persistent reservations.
>
> So, I would allow the scenario of interest (each Initiator port
> connects to only one Target port), but only in the fashion currently
> found in other SCSI transports where every possible connection is
> attempted, and the reasons that not every connection is established
> are external (and configurable, modulo physical constraints).  The
> crucial aspect of this is that if "conservative reuse" does not
> happen (breaking shared persistent reservations), the reason that
> it does not happen MUST be externally configurable as opposed to
> being an unchangeable internal characteristic of the implementation.
>
> With luck, this makes a little more sense.
>
> Thanks,
> --David
>
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA  01748
> +1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
> black_david@emc.com         Cell: +1 (978) 394-7754
> ---------------------------------------------------
>
>
> > -----Original Message-----
> > From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> > Sent: Wednesday, January 02, 2002 4:21 PM
> > To: Black_David@emc.com
> > Cc: ips@ece.cmu.edu
> > Subject: Re: iSCSI: "conservative reuse" requirement
> >
> >
> > Now that I'm back from vacation, let me offer some (delayed)
> > comments on this.  I also read Jim's reply where he appeared
> > to mostly agree with me.
> >
> > >...as long as it presented
> > > each of them to all target ports...
> >
> > I disagree with your assumption that an initiator must
> > "present" *all* its ports to every target port in a target
> > device.  Each distinct initiator port (signified by a unique
> > ISID) may want to talk to only one target port (i.e. one portal
> > group) - that's a typical usage of multi-HBA (i.e. multi-port)
> > hosts talking to multi-FC port targets today.  As far as I
> > can understand, a mandatory "conservative reuse" would
> > preclude this usage.  Please comment if your visualization
> > of conservative reuse is different.
> >
> > OTOH, I can see that the ability of *one* initiator SCSI
> > port to establish nexii with multiple target SCSI ports in
> > a target device is highly desirable (perhaps even T10-required
> > shortly).  IMO, iSCSI had already met this need by defining
> > and enabling "conservative reuse", and that's where we should
> > stop.  I would actually advocate wording along the lines of -
> >
> >     "When a given initiator implementation seeks to present
> >      a single SCSI port abstraction to multiple target portal
> >      groups (thus multiple SCSI target ports) of an iSCSI target
> >      node, conservative reuse is the means to realize it.
> >      Conservative reuse hence MUST be used in this case."
> >
> > and leave it at that.
> > --
> > Mallikarjun
> >
> >
> > Mallikarjun Chadalapaka
> > Networked Storage Architecture
> > Network Storage Solutions Organization
> > MS 5668 Hewlett-Packard, Roseville.
> > cbm@rose.hp.com
> >
> >
> > Black_David@emc.com wrote:
> > >
> > > Not exactly.  "Conservative Reuse" would allow an initiator
> > > to present multiple initiator ports, as long as it presented
> > > each of them to all target ports (assuming that the connectivity
> > > exists).  Why would an Initiator want to present different
> > > ports to different target portal groups?  I don't think there's
> > > another example in which SCSI behaves this way in practice.
> > >
> > > --David
> > >
> > > > -----Original Message-----
> > > > From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> > > > Sent: Monday, December 24, 2001 12:47 AM
> > > > To: Black_David@emc.com
> > > > Cc: ips@ece.cmu.edu
> > > > Subject: Re: iSCSI: "conservative reuse" requirement
> > > >
> > > >
> > > > > I think this is headed towards "conservative reuse"
> > being a MUST if
> > > > > we're serious about support for shared persistent reservations.
> > > >
> > > > Mandating "conservative reuse" appears to force initiators to
> > > > always act
> > > > as a single initiator port (wrt one target; assuming only one
> > > > session as an
> > > > example) per initiator device - which rules out the case of
> > > > an initiator
> > > > intentionally wanting to present a different port to each
> > > > target portal group.
> > > >
> > > > IMHO, if iSCSI provides an architected mechanism to support shared
> > > > persistent reservations ("conservative reuse"),  that should
> > > > be completely
> > > > adequate to meet the expectations to be a legal SCSI protocol.
> > > > --
> > > > Mallikarjun
> > > >
> > > > Mallikarjun Chadalapaka
> > > > Networked Storage Architecture
> > > > Network Storage Solutions Organization
> > > > Hewlett-Packard MS 5668
> > > > Roseville CA 95747
> > > >
> > > > ----- Original Message -----
> > > > From: <Black_David@emc.com>
> > > > To: <santoshr@cup.hp.com>
> > > > Cc: <ips@ece.cmu.edu>
> > > > Sent: Friday, December 21, 2001 4:50 PM
> > > > Subject: iSCSI: "conservative reuse" requirement
> > > >
> > > >
> > > > > Santosh Rao writes:
> > > > >
> > > > > > I am opposed to the suggestion that "conservative re-use"
> > > > of ISIDs be
> > > > > > made a MUST. This is really only required when initiators
> > > > need to be
> > > > > > using the new T10 Reservation scheme that can be shared
> > > > > > across initiator ports.
> > > > >
> > > > > Those reservations are a Target feature.  With this
> > > > approach, the ability
> > > > > to use the target feature depends on details of the initiator
> > > > > implementation.
> > > > > More below ...
> > > > >
> > > > > > For those initiators that don't care about this type of
> > > > reservation,
> > > > > > conservative re-use is of no use and initiators may
> > like to assign
> > > > > > ISID's in a per-initiator node fashion, thereby, being
> > > > able to use these
> > > > > > ISIDs as a lookup index for the sessions on that
> > initiator node.
> > > > > >
> > > > > > Hence, I suggest that "conservative re-use" be worded as
> > > > > > "encouraged to use" or something to that effect, but
> > not MUST USE.
> > > > > >
> > > > > > Comments ?
> > > > >
> > > > > The "initiator" is more than one entity.  The iSCSI HBA/NIC
> > > > and driver
> > > > > doesn't know whether shared persistent reservations are
> > > > being used and
> > > > > shouldn't have to care - they're just more SCSI commands to
> > > > transport.
> > > > > Some other entity (e.g., clustering software) will be
> > generating the
> > > > > shared persistent reservations.  This raise the
> > possible scenario
> > > > > involving a target that supports the new shared persistent
> > > > reservations
> > > > > and an entity that wants to use them.  The entity detects
> > > > (via SCSI means,
> > > > > e.g., something in a mode page) that the Target supports
> > > > shared persistent
> > > > > reservations, tries to use them, only to discover that they
> > > > don't work
> > > > > because the iSCSI HBA/NIC doesn't implement
> > "conservative reuse".
> > > > >
> > > > > I'm worried about this causing both interoperability issues
> > > > and possible
> > > > > T10 issues -- from a T10 viewpoint, if shared persistent
> > > > reservations
> > > > > don't work, the initiating entity should have some
> > SCSI-level means
> > > > > of determining this ... if that means exists only on
> > the Target, the
> > > > > above scenario is iSCSI's problem (Target can't query
> > Initiator to
> > > > > determine whether it does "conservative reuse"), and having
> > > > a separate
> > > > > initiator side means that the entity has to check only for
> > > > iSCSI (and
> > > > > not for any other SCSI transport) does not seem like the right
> > > > > approach.
> > > > >
> > > > > I think this is headed towards "conservative reuse"
> > being a MUST if
> > > > > we're serious about support for shared persistent reservations.
> > > > >
> > > > > Comments?
> > > > > --David
> > > > > ---------------------------------------------------
> > > > > David L. Black, Senior Technologist
> > > > > EMC Corporation, 42 South St., Hopkinton, MA  01748
> > > > > +1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
> > > > > black_david@emc.com         Cell: +1 (978) 394-7754
> > > > > ---------------------------------------------------
> > > > >
> > > > >
> > > >
> >
>