Re: iSCSI Autosense Consensus, Connection next steps

[Stephen Bailey <steph@cs.uchicago.edu>]

> About 1 year ago we had this "transport bundling" aka load sharing IN
> SCTP. At the request of the IESG it was removed because (if I remember
> right) it causes all sorts of problems with Congestion Control and
> is still deemed a research topic by the IETF (belonging to the
> IRTF).

Interesting.  This speaks volumes.

I think more than anything else this goes to the heart of the `it
won't work' claim.  I have claimed that connection bundling support
should either:
  a) be in the transport (or a low network layer), because layers
     above the transport (ULPs) do not have effective control over
     network path.
or 
  b) be in the highest possible layer, because the low level
     components can not be engineered to provide the desired behavior
     without compromise, so ultimately the user must be exposed to, 
     aware of, and control the nature of the compromises.

> It is aong the same lines (to a small degree) as to why you don't
> want multiple TCP connections.

Very roughly.  The specific problem is that NONE of the layers in an
IP end-point have complete control over path selection.  The IP layer
has the best control over this, and it only controls the initial
interface selection.  There's no robust way to know that outside of
the first (and possibly last) hop, you're actually using separate
paths.  If there is a shared path component, bundled connections
through different interfaces will mess with congestion control in the
same way as two connections between the same end-points.

This doesn't mean that it's impossible to control path selection in
ALL cases.  It's just that you have to carefully control the network
configuration in addition to the software behavior.

This is an issue even in the fault tolerance application.  You can not
say ``all you need to do to get fully redundant paths is hook two NIC
cards to each endpoint.''  If the intervening network is beyond your
control, you have no idea, and no way to tell what multipathing you're
actually getting.

The question is whether there should be a feature in iSCSI which
breaks if and when lower layer path selection logic changes.  I am
arguing that a clean separation of concerns between iSCSI and the
lower layers on path selection will make iSCSI a more durable
protocol.

> But the actual implemenation of the load balancing is above the
> transport layer... :0

This has to be because IETF doesn't have an acceptable solution, so
they're punting.  They say that connection bundling messes with
congestion control, and then the proceed to hand the responsibility
over to layers that are completely ignorant of congestion.

Pushing this function up above the transport is not likely to
meaningfully change the nature of the traffic flow at all.  It will
ensure that there are numerically fewer ULPs actually using it, but
that's about it.  A ULP solution still has the same potential failure
modes as that same ULP using a transport level solution.

Nonetheless, it certainly has been argued here that iSCSI is a ULP
that needs this function.  If people really think that the reality
that the feature may or may not work on a configuration by
configuration basis is OK, and an iSCSI-specific solution offers
something important anything over the wedge driver solution, hey, rock
on.

Steph