SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: ISID and the New Persistent Reservations Proposals



    
    David,
    
    I think I'm finally coming around to understand the full nature of what
    you're proposing.
    
    However, I'm not comfortable with it, though I have no more strong
    technical arguments against it.
    
    My strongest discomfort is this:  in essence you are removing the SCSI
    Initiator Port Name from the iSCSI implementation of the model.  You're
    hoping to fill that gap later with a text key whose semantics have yet to
    be defined.  My expectation is that once you attempt to put the name back
    in you're recreated the OptionA/OptionB debate and will never be able to
    solve that satisfactorially.   I'm concerned that removing the notion of
    SCSI Initiator Port Name from the model today, you will have strongly
    hindered iSCSI's ability to take advantage of a number of things that T10
    will be able to do,  now that Names are part of the lexicon.   I also
    believe as I've stated that the OS's are going to prefer to see a more
    stable SCSI port view of the initiator's ports than we're currently
    anticipating in iSCSI.  Finally, I think
       - the reason to do this rather than the ISID is that it cleanly splits
       persistent reservation management away from iSCSI session management.
    is a major violation of layering.  You're going to burden the reservation
    management with having to coordinate iSCSI actions.  [I've argued that
    conservative reuse of ISIDs to different target portal groups provides a
    clean consistent view of SCSI Initiator Ports to the upper layers and
    that's all that's needed.]
    
    None of those are strong technical arguments however.
    
    I think we agree that the problem comes up because the belief that
    coordination of ISIDs on the initiator side is going to be difficult.  I'm
    not convinced that's the case, but let me try a different approach, which
    may make the implementation easier.
    
    Let's postulate two things (a) that initiator portal groups can get their
    iSCSI Name from the OS (without that, the issue is moot because theyn each
    portal group would have its own name and its management of ISIDs) and (b)
    that the initiator portal groups can be enumerated by the OS (as they are
    enabled/discovered/installed/etc.) (/dev/iscsi0,...).  In other words, each
    initiator portal group would have access to the iSCSI Name and it's portal
    group tag.  [You outlined a similar thing on the target side.]
    
    We have two choices:
    (a) embed the initiator portal group tag in the ISID (that provides
    effective partitioning without management intervention).
    (b) put the initiator portal group tag in a text key (then I don't need to
    partition ISIDs, as each will be qualified by the tag.
    
    These are semantically equivalent (it's just as if the ISID field got
    bigger in (b)).  The only reason to choose option (b) over (a) then is if
    the ISID field (2bytes) isn't big enough.
    
    You've suggested that (a) is a "preferred implementation" for the target
    side.  Is there any reason that can't be done in the same way on the
    initiator side? All devices get enumerated in some form or another in every
    OS.  We can then take advantage of that inherent enumeration.
    
    Some final points.
    
    You wrote:
      As I indicated earlier, ISIDs in their current form aren't sufficient
      to deal with proposal #1 - while the specification will map onto iSCSI,
      the current specification of ISIDs will not ensure that the added
      functionality will work as intended.  We would have to significantly
      expand the ISID rule to do so (that expansion will be problematic),
      and hence if proposal #1 is adopted, we may wind up with not only
      the ISID, but also a text key to deal with the resulting mismatch.
    I don't understand these claims still.   Why aren't ISIDs sufficient with
    conservative reuse (as noted above).  What other expansion to ISID rule do
    you foresee?  As I've said, there must be a rule that prevents parallel I_T
    nexus and that will have to be based on the SCSI Initiator Port name
    (however that is defined).  If you don't use the ISID as part of the name
    but a key (or both), then you'll get a new rule with exactly the same
    language, just replacing ISID with "key" or "key + ISID".
    If you don't provide a SCSI Initiator Port name, you will never be able to
    enable proposal #1 (and you'll look an awful lot like parallel SCSI in
    functionality).
    
    I'd like to hear from some of the other T10 folks on this list, if any have
    opinions on this issue.  After that, I'd suggest we call concensus as I've
    run my fingers ragged (and my brain as well).
    
    Jim Hafner
    
    
    Black_David@emc.com@ece.cmu.edu on 10/05/2001 07:43:04 am
    
    Sent by:  owner-ips@ece.cmu.edu
    
    
    To:   John Hufferd/San Jose/IBM@IBMUS, ips@ece.cmu.edu
    cc:
    Subject:  RE: iSCSI: ISID and the New Persistent Reservations Proposals
    
    
    
    John,
    
    Thanks, that's a useful summary.  There are some important points to add.
    
    > However, proposal #1 needs a unique ISID, and can not use a dynamically
    > created SSID generated by each target Port.
    
    Or it needs an iSCSI text key to serve the port naming function of the ISID
    - the reason to do this rather than the ISID is that it cleanly splits
    persistent reservation management away from iSCSI session management.
    This has a couple of advantages:
    - It allows us to exactly accommodate whatever T10 does by defining the
         semantics of the text key appropriately.  We won't have the same
         degree of flexibility in redefining ISID behavior.
    - It allows us to resolve the "blow it away" issue caused by ISID reuse
         without having to trade off aspects of that resolution against
         consequences for proposal #1, whose status in T10 is unknown.
    I would prefer deferring this whole issue of "New Persistent Reservation
    Proposals" until T10's direction is clear, and in the interim advise
    T10 that proposal #1 is a poor fit to iSCSI.
    
    As I indicated earlier, ISIDs in their current form aren't sufficient
    to deal with proposal #1 - while the specification will map onto iSCSI,
    the current specification of ISIDs will not ensure that the added
    functionality will work as intended.  We would have to significantly
    expand the ISID rule to do so (that expansion will be problematic),
    and hence if proposal #1 is adopted, we may wind up with not only
    the ISID, but also a text key to deal with the resulting mismatch.
    This is one more reason to stop trying to design in anticipation of
    what T10 may or may not do.
    
    > Now you also have to ensure that upon a Target Boot, (which has to
    remember
    > the Persistent Reservation Status), it needs to also remember the high
    > water mark of ISIDs given out to each Initiator Node.  With that high
    water
    > mark the Target can continue assigning the ISIDs for that Initiator Node.
    > Then I think your words were do not aggressively reused ISIDs.  I think
    > that if the Target has not gone down for a day or so, ISIDs not in use
    > might be re-used, etc.
    >
    > We have 65K valid values per Initiator Node so reuse should not be a
    problem.
    
    This is actually a "SHOULD", not a "MUST", because a properly coded
    Initiator
    can figure out what's going on even if TSIDs (ISIDs in the above paragraph)
    are reused with some care.  I'll spare everyone the details, as they're a
    bit
    subtle and deal with a situation in which the Target has already made a
    mistake
    (lost a persistent reserve), and hence it's reasonable to wonder what else
    the Target will get wrong.  We agree that it's better to avoid depending on
    completely correct behavior in this sort of error situation.
    
    > So anyway, as good engineers we have been able to come up with something
    > that works.  The problem that the N & D team had is, so what.  What we
    have
    > now works, and has a smaller impact on the Target.
    
    I'm not sure that what we have now "works" courtesy of the Option A/Option
    B
    issue on when a Target is supposed to blow away a previous session.  The
    current requirement that the Initiator specify the ISID, along with the
    expectation that ISID values will be aggressively reused is the direct
    cause
    of that problem.  The best solution I've seen to it is to remove ISIDs,
    yielding a clear distinction between an Initiator who specifies a TSID
    (wants its old state back) vs. sends 0 as the TSID (wants a brand new
    session).
    
    I also take issue with the "smaller impact on the Target" statement
    above -- it may be based on the incorrect assumption that Targets would
    have to manage ISIDs in addition to TSIDs.  ISIDs can be removed in their
    entirety, and TSID management would continue to work in essentially the
    current fashion.  There's been at least one statement on the list that
    having Targets completely control the allocation of identifiers used to
    manage target resources (e.g., persistent reserves) simplifies Targets.
    
    > I had sent a note in previously that explained that if we have a way to
    set
    > the iSCSI Initiator Node Name then we must have a way to set the ISID by
    > the Initiator Node.  Therefore, the question goes, why get the Target
    > involved.
    
    Wrong question, IMHO.  Try "If TSIDs already solve the problem, why get the
    Initiator involved?" :-).  Those who want to keep the ISID need to defend
    the additional complexity that it adds to the protocol, rather than
    uniformly
    criticize any proposed change as disruptive, IMHO.
    
    As to the statement that the ISID can be configured to an HBA in the same
    manner as the iSCSI Initiator Node Name, I observe that fewer things to
    configure lead to fewer opportunities to make mistakes and better scaling
    of management.
    
    > Therefore, additional routines on the Target for handing out ISIDs should
    > not be needed.
    
    I agree, just get rid of ISIDs entirely :-).
    
    --David
    
    
    > -----Original Message-----
    > From: John Hufferd [mailto:hufferd@us.ibm.com]
    > Sent: Thursday, October 04, 2001 8:52 PM
    > To: Black_David@emc.com
    > Cc: ips@ece.cmu.edu
    > Subject: iSCSI: ISID and the New Persistent Reservations Proposals
    >
    >
    > David,
    > I have spent some time stepping through the issues of Target Generation
    of
    > SSID or ISID along with Persistent Reserves.  I will start with
    Persistent
    > Reserves and then move to the SSID/ISID issue.
    >
    > I have tried to lay it out persistent reserves so that everyone can
    follow
    > it (may not work out but that is the attempt), I have also reviewed this
    > with Jim Hafner, so unless I have some typos I think it is correct.
    >
    > Persistent Reserves have three new proposed versions: (in each of the
    > following assume that the Initiator Node has 2 Ports A, and B, and the
    > Target Node has 2 Ports X, and Y. Also the subject LUN is called LU5).  I
    > will start with point #0 that represents Persistent Reservations as of
    > today, and then step through the 3 new options.
    >
    > 0. Today, Initiator Port A, can create a Persistent Reservation for LU5
    > with Target Port X, and then only I/O to LU5 can occur via the path from
    > Initiator port A to Target Port X, all other I/Os to LU5 will be blocked.
    > That is LU5 I/O via A to Y will be blocked as will B to X and B to Y.
    >
    > Three New additional proposals for handling Persistent reservations are
    > being pushed by folks in T10:
    > 1. Persistent reservations to LU5 can be made via Port A, and that will
    > apply on the A to X path and the A to Y path.   I/O to LU5 from Port B to
    > either X or Y will be blocked.
    > 2. Persistent reservation to LU5 made via Port A  or Port B to Target
    Port
    > X, will permit  I/O to LU5 via Port A or B, if sent to Target Port X.
    > However, I/O to LU5 via Initiator Ports A or Port B will be blocked if it
    > is sent to Target Port Y.
    > 3. Persistent reservation to LU5 made via Initiator Port A or Port B to
    > Target Ports X or Y may occur, and I/O to LU5 may then occur via any path
    A
    > to X, A to Y, B to X, or B to Y.
    >
    > In both todays model (#0) or in proposal #1 the Target needs to place the
    > Persistent  reservation against the Full iSCSI Initiator Port Name.  This
    > is the iSCSI Initiator Node Name Plus the ISID.  In proposal #1 the Full
    > iSCSI Initiator Port Name is then made known to all the target ports (in
    > the same SCSI Device), and that name is then used to determine the
    > Persistent Reservation State for each of the Target Ports relative to the
    > subject LU.
    >
    > In proposal 2 and 3, above,  the name to be used in the Reservations is
    > just the iSCSI Initiator Node Name (No ISID).  In proposal #2 the Name is
    > not shared with the other Target Ports (in that SCSI Device) so
    Persistence
    > only apples to I/O arriving at that Target Port from any Initiator port
    on
    > the Initiator Node which made the Reservation.  And in proposal 3 the
    iSCSI
    > Initiator Node Name is shared among the Target Ports so that Persistence
    > can be applied to any Target Port (in that SCSI Device), which is
    addressed
    > by any Port on the Initiator Node which made the Reservation.
    >
    > Now, with regards to the issue that has been kicking around about the
    > target assigning the SSID,  it does not matter one way or the other with
    > today's position #0 or proposal numbers 2 or 3. where the SSID comes
    from.
    > However, proposal #1 needs a unique ISID, and can not use a dynamically
    > created SSID generated by each target Port.  There are other reasons why
    > folks do not even want the ISID, or the SSID generated by the target, but
    > those reasons have nothing to do with these Persistent Reservation
    > proposals.
    >
    > I have been suggesting to the N &D team and others that if we limit the
    > Dynamic Target Allocation to only the ISID (and of course the TSID) but
    not
    > a complete SSID, that there may be a workable solution.  I say ISID,
    since
    > I believe it must be assured uniqueness per Initiator Port.  One reason
    can
    > be seen in #0, & #1 above.
    >
    > Now suppose that one of the Target's Job was to Generate a unique ISID
    for
    > each Initiator Port, within and Initiator Node, which wanted to go into
    > session.  If you combine with that, the rule that if the Initiator Port
    > sets and sends its own ISID  on the Login the Target will accept it, and
    > then use the Full Initiator Port Name to check for Persistent Reserves, I
    > think we approach a working solution.
    >
    > Now you also have to ensure that upon a Target Boot, (which has to
    remember
    > the Persistent Reservation Status), it needs to also remember the high
    > water mark of ISIDs given out to each Initiator Node.  With that high
    water
    > mark the Target can continue assigning the ISIDs for that Initiator Node.
    > Then I think your words were do not aggressively reused ISIDs.  I think
    > that if the Target has not gone down for a day or so, ISIDs not in use
    > might be re-used, etc.
    >
    > We have 65K valid values per Initiator Node so reuse should not be a
    problem.
    >
    > OK now lets review what we have said.  There is a way for a very simple
    > routine on the Target to assign the ISIDs.  If Sent an ISID in the Login
    it
    > should accept it (if it can), and check on persistent reserves of type
    #0,
    > or #1.  The Initiator can save the ISID, in case it goes down and want to
    > come back up, and reclaim the reserves, or it can blow the reserves away
    > and start again with a Target assigned ISID.
    >
    > OK now in one of your last notes, I think you said a similar thing.  This
    > is also what I attempted to explain to the N &D team.
    >
    > So anyway, as good engineers we have been able to come up with something
    > that works.  The problem that the N & D team had is, so what.  What we
    have
    > now works, and has a smaller impact on the Target.
    >
    > So if it works we need to decide if we should do it or NOT, and focus the
    > conversation on that.
    >
    > I had sent a note in previously that explained that if we have a way to
    set
    > the iSCSI Initiator Node Name then we must have a way to set the ISID by
    > the Initiator Node.  Therefore, the question goes, why get the Target
    > involved.
    >
    > Here is what I said in another note:
    >
    > If you accept that HBAs should get the iSCSI Initiator Node Name passed
    to
    > it in some manner from the OS, then we did not see why it can not get the
    > ISID at the same time also.  If one were to argue that the HBA can not
    get
    > the iSCSI Initiator Node Name, I think this is a bit strange, since
    almost
    > every thing I install in systems, has to go through some type of Wizard,
    or
    > vendor install routine  that actually installs the support driver, etc.
    > (This is all set when the OS is fully up and the Registry etc. is clearly
    > available.)  I would expect that with current OSs, this would be a normal
    > approach.  In the future, I suppose it might be possible that  the OSs
    > might have a more dynamic routine to hand out the iSCSI Initiator Node
    Name
    > and ISID so that Hot Swap HBAs could come up without effort.  However,
    > until then, I am not sure it is unreasonable for a vendor to require the
    > customer to run an update routine,  even after performing an HBA Hot
    Swap.
    >
    > There is of course, the first up boot situation.  I think it should be
    > possible to have a default, first time HBA boot Name that uses the iqn
    form
    > of the iSCSI Name, and an ISID of FFFF (not 0 as said previously).  This
    > should be enough to bring up a "first time up boot", since after the OS
    is
    > started, the approprate install routine should be kicked off, and the
    real
    > iSCSI Initiator Node Name and real ISID set in the Flash for that, and
    any
    > other iSCSI HBAs.
    >
    > By the way I have a SCSI HBA in my personal system that runs its own BIOS
    > routine and when it comes up, but before it boots, it asks questions if
    > needed or just announces it is there.  Configuration value can be entered
    > at that time.  So even that approach is possible.
    >
    > So based on the above, I do not think requiring the HBAs to have an
    > install/update routine (which sets its Flash) is even a problem.
    >
    > This means that a Single iSCSI Initiator Node Name should be easy to
    obtain
    > and will apply for all the iSCSI Initiators, whether HW or SW. And
    likewise
    > it should be straight forward to secure the ISID at the same time.  All
    > this following well understood current install processes.
    >
    > Therefore, additional routines on the Target for handing out ISIDs should
    > not be needed.
    >
    > OK, I tried to bring all the ISID issues together here so we could all
    > examine them.  I hope this was useful.
    >
    >
    >
    >
    >
    >
    >
    >
    > .
    > .
    > .
    > John L. Hufferd
    > Senior Technical Staff Member (STSM)
    > IBM/SSG San Jose Ca
    > Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
    > Home Office (408) 997-6136
    > Internet address: hufferd@us.ibm.com
    >
    
    
    
    


Home

Last updated: Fri Oct 05 23:17:23 2001
7086 messages in chronological order