RE: iSCSI: "conservative reuse" requirement

[Black_David@emc.com]

This topic is not exactly amenable to short summaries ...

> I disagree with your assumption that an initiator must 
> "present" *all* its ports to every target port in a target
> device.  Each distinct initiator port (signified by a unique 
> ISID) may want to talk to only one target port (i.e. one portal 
> group) - that's a typical usage of multi-HBA (i.e. multi-port) 
> hosts talking to multi-FC port targets today.  As far as I 
> can understand, a mandatory "conservative reuse" would 
> preclude this usage.  Please comment if your visualization
> of conservative reuse is different.

The concern I have is based on the source of the connectivity
constraints.  To begin with, I disagree with the notion that
a FC imitator port "may want to talk to only one target port";
Fibre Channel does not work that way because the connectivity
constraints are imposed externally to the ports.  For the
case of an m-way HBA host talking to a n-way FC port target,
"conservative reuse" is attempted by the HBAs (each port has
exactly one WWN), and if less than the full m x n set of connections
results, it is due to an external absence of connectivity -
either each HBA port is connected to one target FC port, or
the fabric(s) involved are zoned in a fashion that this is
the resulting logical connectivity.  In both cases, an HBA
following the rules connects to everything that it is allowed
to connect to. 

I have no problem with external physical or logical (e.g., VLAN,
or even iSNS) connectivity constraints preventing all possible
connections between a multi-port initiator and a multi-port target.
What I have a problem with is an implementation making an
internal decision that it will *never*:

      present a single SCSI port abstraction to multiple target
      portal groups (thus multiple SCSI target ports) of an iSCSI
      target node,

no matter what its administrators and users may want.  IMHO,
that's not the implementation's decision to make - it needs to
be made externally by the administrator who configures the
system. In particular, if an implementation is operating over
a single IP interface, there has been discussion on this list
of using a different ISID for every target portal group that
it connects to - shared persistent reservations will never work
right if that is done, and there is nothing that the administrator
can do in that situation to get them to work ... I think this needs
to be prohibited if we're serious about following T10's direction
on persistent shared reservations, unless T10 is going to
provide a way to query the initiator to discover that it does
not support shared persistent reservations.

So, I would allow the scenario of interest (each Initiator port
connects to only one Target port), but only in the fashion currently
found in other SCSI transports where every possible connection is
attempted, and the reasons that not every connection is established
are external (and configurable, modulo physical constraints).  The
crucial aspect of this is that if "conservative reuse" does not
happen (breaking shared persistent reservations), the reason that
it does not happen MUST be externally configurable as opposed to
being an unchangeable internal characteristic of the implementation.

With luck, this makes a little more sense.

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
black_david@emc.com         Cell: +1 (978) 394-7754
---------------------------------------------------


> -----Original Message-----
> From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> Sent: Wednesday, January 02, 2002 4:21 PM
> To: Black_David@emc.com
> Cc: ips@ece.cmu.edu
> Subject: Re: iSCSI: "conservative reuse" requirement
> 
> 
> Now that I'm back from vacation, let me offer some (delayed)
> comments on this.  I also read Jim's reply where he appeared 
> to mostly agree with me.
> 
> >...as long as it presented
> > each of them to all target ports...
> 
> I disagree with your assumption that an initiator must 
> "present" *all* its ports to every target port in a target
> device.  Each distinct initiator port (signified by a unique 
> ISID) may want to talk to only one target port (i.e. one portal 
> group) - that's a typical usage of multi-HBA (i.e. multi-port) 
> hosts talking to multi-FC port targets today.  As far as I 
> can understand, a mandatory "conservative reuse" would 
> preclude this usage.  Please comment if your visualization
> of conservative reuse is different.
> 
> OTOH, I can see that the ability of *one* initiator SCSI 
> port to establish nexii with multiple target SCSI ports in 
> a target device is highly desirable (perhaps even T10-required
> shortly).  IMO, iSCSI had already met this need by defining 
> and enabling "conservative reuse", and that's where we should 
> stop.  I would actually advocate wording along the lines of -
>     
>     "When a given initiator implementation seeks to present 
>      a single SCSI port abstraction to multiple target portal 
>      groups (thus multiple SCSI target ports) of an iSCSI target 
>      node, conservative reuse is the means to realize it.
>      Conservative reuse hence MUST be used in this case."
> 
> and leave it at that.
> -- 
> Mallikarjun 
> 
> 
> Mallikarjun Chadalapaka
> Networked Storage Architecture
> Network Storage Solutions Organization
> MS 5668	Hewlett-Packard, Roseville.
> cbm@rose.hp.com
> 
> 
> Black_David@emc.com wrote:
> > 
> > Not exactly.  "Conservative Reuse" would allow an initiator
> > to present multiple initiator ports, as long as it presented
> > each of them to all target ports (assuming that the connectivity
> > exists).  Why would an Initiator want to present different
> > ports to different target portal groups?  I don't think there's
> > another example in which SCSI behaves this way in practice.
> > 
> > --David
> > 
> > > -----Original Message-----
> > > From: Mallikarjun C. [mailto:cbm@rose.hp.com]
> > > Sent: Monday, December 24, 2001 12:47 AM
> > > To: Black_David@emc.com
> > > Cc: ips@ece.cmu.edu
> > > Subject: Re: iSCSI: "conservative reuse" requirement
> > >
> > >
> > > > I think this is headed towards "conservative reuse" 
> being a MUST if
> > > > we're serious about support for shared persistent reservations.
> > >
> > > Mandating "conservative reuse" appears to force initiators to
> > > always act
> > > as a single initiator port (wrt one target; assuming only one
> > > session as an
> > > example) per initiator device - which rules out the case of
> > > an initiator
> > > intentionally wanting to present a different port to each
> > > target portal group.
> > >
> > > IMHO, if iSCSI provides an architected mechanism to support shared
> > > persistent reservations ("conservative reuse"),  that should
> > > be completely
> > > adequate to meet the expectations to be a legal SCSI protocol.
> > > --
> > > Mallikarjun
> > >
> > > Mallikarjun Chadalapaka
> > > Networked Storage Architecture
> > > Network Storage Solutions Organization
> > > Hewlett-Packard MS 5668
> > > Roseville CA 95747
> > >
> > > ----- Original Message -----
> > > From: <Black_David@emc.com>
> > > To: <santoshr@cup.hp.com>
> > > Cc: <ips@ece.cmu.edu>
> > > Sent: Friday, December 21, 2001 4:50 PM
> > > Subject: iSCSI: "conservative reuse" requirement
> > >
> > >
> > > > Santosh Rao writes:
> > > >
> > > > > I am opposed to the suggestion that "conservative re-use"
> > > of ISIDs be
> > > > > made a MUST. This is really only required when initiators
> > > need to be
> > > > > using the new T10 Reservation scheme that can be shared
> > > > > across initiator ports.
> > > >
> > > > Those reservations are a Target feature.  With this
> > > approach, the ability
> > > > to use the target feature depends on details of the initiator
> > > > implementation.
> > > > More below ...
> > > >
> > > > > For those initiators that don't care about this type of
> > > reservation,
> > > > > conservative re-use is of no use and initiators may 
> like to assign
> > > > > ISID's in a per-initiator node fashion, thereby, being
> > > able to use these
> > > > > ISIDs as a lookup index for the sessions on that 
> initiator node.
> > > > >
> > > > > Hence, I suggest that "conservative re-use" be worded as
> > > > > "encouraged to use" or something to that effect, but 
> not MUST USE.
> > > > >
> > > > > Comments ?
> > > >
> > > > The "initiator" is more than one entity.  The iSCSI HBA/NIC
> > > and driver
> > > > doesn't know whether shared persistent reservations are
> > > being used and
> > > > shouldn't have to care - they're just more SCSI commands to
> > > transport.
> > > > Some other entity (e.g., clustering software) will be 
> generating the
> > > > shared persistent reservations.  This raise the 
> possible scenario
> > > > involving a target that supports the new shared persistent
> > > reservations
> > > > and an entity that wants to use them.  The entity detects
> > > (via SCSI means,
> > > > e.g., something in a mode page) that the Target supports
> > > shared persistent
> > > > reservations, tries to use them, only to discover that they
> > > don't work
> > > > because the iSCSI HBA/NIC doesn't implement 
> "conservative reuse".
> > > >
> > > > I'm worried about this causing both interoperability issues
> > > and possible
> > > > T10 issues -- from a T10 viewpoint, if shared persistent
> > > reservations
> > > > don't work, the initiating entity should have some 
> SCSI-level means
> > > > of determining this ... if that means exists only on 
> the Target, the
> > > > above scenario is iSCSI's problem (Target can't query 
> Initiator to
> > > > determine whether it does "conservative reuse"), and having
> > > a separate
> > > > initiator side means that the entity has to check only for
> > > iSCSI (and
> > > > not for any other SCSI transport) does not seem like the right
> > > > approach.
> > > >
> > > > I think this is headed towards "conservative reuse" 
> being a MUST if
> > > > we're serious about support for shared persistent reservations.
> > > >
> > > > Comments?
> > > > --David
> > > > ---------------------------------------------------
> > > > David L. Black, Senior Technologist
> > > > EMC Corporation, 42 South St., Hopkinton, MA  01748
> > > > +1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
> > > > black_david@emc.com         Cell: +1 (978) 394-7754
> > > > ---------------------------------------------------
> > > >
> > > >
> > >
>