RE: iSCSI: "Wedge" drivers

[julian_satran@il.ibm.com]

David,

I am aware of the complexity related to recovery and load balancing (as I
had to design it
in several systems). There are several  points to be made:

- the view used by all the I/O software above channel in the mainframe
world is extremely simple (no wedge driver) as all they see is one
connection

- recovery is also simple as the commands are restarted (no need to track
data transfers)

- load balancing is also simple - it is based on a naive greedy scheme -
take the first
 available channel with some randomness inserted at search start

I attempted this approach in the 00 draft (Adelaide) and we changed it to
the current
draft (with symmetric connections) as most of my colleagues felt that
asymmetric connections will be hard to understand and implement properly.

While many of you keep telling me that wedge drivers won't go away nobody
gave one good example to support that assertion.

I think that as SCSI moves to a level of complexity that goes beyond what
was
available on mainframes (with the standardization of 3rd party commands and
the object
model) it would be a mistake not to attempt to simplify SCSIs view of the
transport and
remove recovery and transport load balancing from it.

Julo

Black_David@emc.com on 28/08/2000 19:24:06

Please respond to Black_David@emc.com

To:   ips@ece.cmu.edu
cc:    (bcc: Julian Satran/Haifa/IBM)
Subject:  RE: iSCSI: "Wedge" drivers




Still with my co-chair hat off, let me try to summarize,
rather than quoting and responding:

If one assumes that an iSCSI session spans multiple
interface processors on the storage side, then those
interface processors have to share SCSI state (e.g.,
ranges of data for each I/O on which Ready to Transfer
has been vs. needs to be issued) in some fashion so
that the other processor has it when failure causes
it to become involved.  Use of a load balancing switch
does not change this situation.

Julian (IIRC) has observed that the corresponding state
sharing is done for mainframe storage (e.g., I/O completion
can come back on a different link than the one used for
initiation), but it is not done by any SCSI implementation
I'm aware of.  If nothing else, that's proof by example
that any box (EMC, IBM, etc.) that supports mainframe
storage is in principle capable of spreading a SCSI
connection across more than one interface processor.
This is additional implementation complexity and effort,
as state sharing across failure domains is not easy to
get right, and this is a new form of sharing that SCSI
implementers have little experience with.

A similar issue occurs on the host side.  If the system
is equipped with NICs so that iSCSI above TCP/IP is entirely
in software, then spreading a SCSI connection across multiple
NICs should work fine.  OTOH, if the system has HBAs that
implement much of iSCSI in hardware in addition to TCP/IP,
then not sharing iSCSI state across HBAs is again easier
to build (and entails a wedge driver).

I'm trying not to take a position for or against iSCSI
sessions (e.g., I can see a strong rationale for using
them with tape, where wedge drivers don't work and the
fault tolerance requirements aren't as stringent).  The
point I wanted to make is that engineering and implementation
concerns make it highly unlikely that iSCSI sessions would
remove the need for wedge drivers for disk.  My goal
is to contribute to a clear understanding of the technical
rationale behind sessions, for the requirements document
if nothing else.

Several minor issues ...

John described a model in which an initiator discovers
that multiple TCP/IP connections are part of the same
session.  That seems awkward by comparison to having
each participant tell the other what the alternate
transport addresses are (or how to find them) as the
latter avoids having to splice a new connection into
one that's already actively doing I/O.  SCTP sets up
multiple addresses in this fashion, FWIW.

EMC error retry does not always require an EMC-written wedge
driver.  For example, Symmetrix works with the Veritas DMP
wedge driver.

There's no free lunch on configuration -- since wedge
drivers are not going to go away, iSCSI sessions become
something configured "in addition to" wedge driver
configuration rather than "instead of".

--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 435-1000 x75140, FAX: +1 (508) 497-6909
black_david@emc.com  Cellular: +1 (978) 394-7754
---------------------------------------------------