Re: iSCSI: "Wedge" drivers

["Randall R. Stewart" <randall@stewart.chicago.il.us>]

John:

I don't think that it has been decided that you are using TCP or SCTP 
as of yet. This issue is still up for debate and I have not yet
sensed a consenses (IMHO) as to which way to go...

TCP does provide reliable delivery and one connection. It will
give you the congestion control you need but it does have 
issues such as message boundaries and multi-homing support.

I have not heard and I guess I have not formally asked... is
the head of line blocking issue at all relevant.

It sounds like you CAN fire off several commands in parallel... 

If you do this, MUST they be delivered in order, or do you 
want them delivered and processed in any order?

Do you want to have the option of NOT retransmitting datagrams
or Douglas's retransmit at most 1 time... I think this will
be added to the u-sctp draft.. it is a neat feature...

Now as to availablility of SCTP.. v9 is now available (you can
get a ref-impl and I know 10 vendors working on supporting SCTP
.. at least 10).

SCTP V13 (the rfc version) will be supported in the ref-impl next
week.. inbetween email, this is what I am working on... almost done
I just need to see what I have broken with a bit of testing :)

So SCTP is out there and is available.. it is not as predominant as
TCP nor do I know of it in silicon.. but on the other hand it adds
a LOT of features that the iSCSI can use...


R



John Hufferd/San Jose/IBM wrote:
> 
> Y.P.
> I thought the first part of your note was useful, but then you went to
> something about an FCP/SCTP and creating an IP-to-FC-address from a
> translation table, etc.  At that point, you diverted from the iSCSI draft
> and went where I do not believe many of us were headed.  The current draft
> assumes we are using TCP/IP not SCTP and has no additional layer of FCP.
> 
> .
> .
> .
> John L. Hufferd
> 
> "Y P Cheng" <ycheng@advansys.com>@ece.cmu.edu on 08/29/2000 06:58:45 PM
> 
> Sent by:  owner-ips@ece.cmu.edu
> 
> To:   Julian Satran/Haifa/IBM@IBMIL, <ips@ece.cmu.edu>
> cc:
> Subject:  RE: iSCSI: "Wedge" drivers
> 
> Julo Wrote:
> >I understand 3 and I somehow expected it. If I would have to
> >reimplement this under iSCSI I would choose a plug-in in
> >iSCSI to do it (a policy module) and iSCSI would clearly
> >support such a construct (and make it also future-proof).
> >I have some trouble with 1 and 2.  In whatever design you
> >choose at the array level you have to have some sharing as
> >some commands are targeted to a LU and all it's queues
> >(i.e. separation per initiator is never complete if you
> >take SAM seriously). However some design might choose to
> >share only for those commands - and talking to you
> >with you chair hat off - I don't think that you are with
> >one of those (:- I am no sure I understand 4 either - if
> >the array does no "sense" the fail over.
> 
> I have been reading the "Wedge Drivers" and their "load balance and fail
> over recovery" with great interest.  I am not familiar with the mainframe
> I/O design.  However, I worked on SCSI and fibre channel host bus adapters
> (HBA's) for long time.  I would like apply those discussions on the
> adapters
> that I know of.  The understanding of the HBA's would greatly simplify the
> discussions of wedge driver and its load balance and recovery.
> 
> The SCSI and fibre channel HBA's today executes a SCSI command atomically
> without any intervention from the device driver.  When there are multiple
> HBAs on one server, they don't share command execution states.  Each HBA
> executes their own SCSI commands atomically.  The HBA and its driver
> perform
> discovery of SCSI or FCP devices at power up.  It numbers the devices
> sequentially.  A table is created to map the device number to a SCSI ID or
> a
> fibre channel 24-bit address.  When there are multiple HBA's, it is the
> responsibility of the Upper Level Drivers to direct the SCSI commands
> through their appropriate HBA's.   The HBA itself can care less.
> 
> For a iSCSI HBA, there won't be any discovery at powerup.  Instead, the HBA
> will monitor the ARP and FARP requests and responses to create a
> IP-to-FC-address translation table. (This discussion does not exclude an
> Ethernet NIC card.)  The HBA relies on the FCP/SCTP driver to create
> endpoints and associates for connection oriented exchanges. The iSCSI
> driver
> running above the FCP/SCTP driver and below the SCSI-class driver will be
> responsible for SCSI-over-IP messages.  In fact the iSCSI HBA is no
> different from an NIC with the exception of the FCP/RDMA support. When
> there
> are multiple iSCSI HBA's, The FCP/SCTP driver will be responsible to
> determine if an IP device can be reached from a different HBA. (Whether an
> IP device can be reached by another IP address is beyond the scope of this
> discussion.)  We may choose to include a subset of the SCTP function in the
> iSCSI driver if we don't wait for the SCTP implementation.  In the
> discussion below we refer the iSCSI driver with a subset of the SCTP
> function.
> 
> Knowing there is an alternative HBA to reach the same IP device, it is
> certainly the discretion of the iSCSI driver to sent IP-SCSI messages to
> the
> HBA of its choice. A response must come back to the same HBA because it is
> difficult for two HBA's to share execution states.  For example, if we set
> up one HBA to do the RDMA function, it is not nice to have the data coming
> back to the other HBA. Both fail over and load balance can be done in iSCSI
> driver relatively easily because it is fully aware of the availability of
> alternative HBA's reaching the same IP device.  In theory, we could break
> up
> the iSCSI driver by moving the SCSI-to-IP-packet conversion function to the
> HBA and by keeping all the remaining functions.  If we do so, the SCSI
> command will be executed atomically inside an HBA.

-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222