Re: iSCSI Autosense Consensus, Connection next steps

[julian_satran@il.ibm.com]

Randal,

OK - the main reason we want to use several connections is link aggregation
(i.e. different
physical connections).

And then even attempting to use more aggressively congestion control will
not give you
an edge on the network (not a significant one) as long as you are using the
same IP address
as I assume that RED is looking for this too (and if not it should!).

An interesting point - in  a former life doing a satellite transmission I
decide against such a trick
and used a FEC scheme to bring BER in-line with the wired links (in fact 1
order of magnitude better to compensate for RTT).

Julo



"Randall R. Stewart" <randall@stewart.chicago.il.us> on 06/09/2000 16:36:38

Please respond to "Randall R. Stewart" <randall@stewart.chicago.il.us>

To:   csapuntz@cisco.com
cc:   ips@ece.cmu.edu, Allison Mankin <mankin@east.isi.edu>, Scott Bradner
      <sob@harvard.edu> (bcc: Julian Satran/Haifa/IBM)
Subject:  Re: iSCSI Autosense Consensus, Connection next steps




Ok,

explain something to me...

csapuntz@cisco.com wrote:
>
> Asymmetric connections may be the way to go.
>
> Let's look at characteristics of control vs. data:
>
> * iSCSI control
>         - low bandwidth (handled on a CPU)
>         - should generally be processed in FIFO order
>         - complex state machines (probably software implemented)
>         - flow control nice
>         - doesn't need to end up in any special buffers
>           (can be processed in-place)
>
> * iSCSI data/ready to transmit (rtt)
>         - potentially high bandwidth
>         - with appropriate headers, segments can be
>           processed entirely out-of-order
>         - simple data transfer state machine
>         - no flow control on data needed, just congestion control
>                    - rtt's need to be flow-controlled, but
>                      that can be done with a simple credit mechanism
>         - data needs to end up in special buffers (e.g. buffer cache)
>
I understand how you will obtain all of the above with
two TCP connecions. I can even support this, considering
the 1 for control is "low bandwidth" (note: I can't support
N data connections.. only 1).

But my question is to the Data connection. How do you get
"no flow control on data needed, just congestion control" on
a TCP connection?


> The processing requirements of the data and control seem quite
> asymmetric. Mixing them on one TCP channel makes separating out the
> data and control more difficult.
>
> Conversely, by putting control and data on separate TCP connections,
> you can use the flow label (IP addresses, TCP ports) to determine
> whether to route this along the control or data path in your machine.
>
> So, I think there is value in the proposal that we use 1 control
> connection and up to n separate dedicated data/RTT connections.

No I can NOT agree with N seperate dedicated data/RTT connections. I
think you should go look at Sally's email that was posted here
earlier... here it is...
**************************************************************************
Franco Travostino wrote:
>
> A bit of Related Work ... On the forever-young 1 vs. N TCP connections
> subject, you may be interested in the results shown in the '97 "An
> Application-Level Solution to TCP's Satellite Inefficiencies" paper
> (available at http://jarok.cs.ohiou.edu/projects/satellite/) and on Sally
> Floyd's comment hereafter. The intersection with satellite links is quite
> accidental, and the arguments/results apply above and beyond satellite
links.
>
> -franco
>
> >From majordom@ISI.EDU Fri Dec  6 23:05:58 1996
> >To: fred@cisco.com (Fred Baker)
> >Cc: end2end-tf@isi.edu
> >Subject: Re: Related paper/re:satellites
> >Date: Fri, 06 Dec 1996 23:05:47 PST
> >From: Sally Floyd <floyd@ee.lbl.gov>
> >Sender: owner-end2end-tf@ISI.EDU
> >Precedence: bulk
> >Content-Length: 2309
> >X-Lines: 44
> >
> >Fred -
> >
> >>You might be interested in reviewing this paper, which is what I'm
> >>discussing with Karen Hansen, Dan Shell, and the folks from Comsat. It
> >>relates to some TCP/Satellite work being done at NASA Lewis Research
> >>Center.
> >
> >Basically (from a quick read), the paper, on "An Application-Level
> >Solution to TCP's Satellite Inefficiencies", recommends breaking a
> >single TCP connection into multiple connections at the
> >application level, to increase throughput on satellite circuits.
> >It is not too surprising that this increases the TCP throughput, but it
> >is still not a good idea.  For a single TCP connection, a single packet
> >drop results in the throughput for that connection being cut by half,
> >and then increased by roughly one packet per RTT.  For a TCP connection
> >that has instead been separated into N different TCP connections, a
> >single packet drop results in one of the N connections, receiving
> >1/N-th of the total bandwidth, having its throughput cut in half.  Thus
> >the bandwidth of the aggregate connection has its bandwidth reduced to
> >(1 - 1/(2N))-th of its former bandwidth - that is, the larger the value
> >for N, the smaller the aggregate bandwidth is cut.  And then, because
> >each TCP connection continues to increase its congestion window by one
> >packet per RTT, for those TCP connections that have not yet reached the
> >receiver's advertised window, the aggregate TCP connections together
> >increase their bandwidth by up to N packets per RTT.
> >
> >Summarizing, splitting a TCP connection into N separate connections
> >simply increases the aggressiveness of the TCP congestion control.
> >Meaning that this TCP is now more likely to "steal" bandwidth from
> >other TCP connections in times of congestion.  And increasing the
> >aggressiveness of the TCP congestion control too far (by choosing N too
> >large) is counterproductive even for the aggregate TCP connection, as
> >the paper shows.
> >
> >I would suggest that this is exactly the kind of development for which
> >[RED is needed].
> >
> >- Sally
> >
> >
> >
******************************************************************
In my mind it will be a hard sell to the IESG to ask for
multiple connections for high-bandwidth data? Scott/Allision any
comments on this?

>
> The symmetric, connection allegiance case was developed when we were
> smoking some really good stuff in Haifa. It really addresses a
> corner case: minimizing the communication needs between NICs in systems
> that will stripe requests across NICs.
>
> However, somewhere along the line, we forgot to optimize for the
> single NIC case.
>
> ------------------
>
> Also using a separate TCP connections may allow us to migrate one
> day to an architecture where a different node in the IP network
> returns the actual data from a SCSI READ!
>
> Client <---- iSCSI Control ----> Storage Controller <---- TBD ----> Disk
array
>   ^                                                                    ^
>   +-------------------------------iSCSI Data protocol -----------------+
>
> -Costa

--
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222 (cell) 815-477-2127 (work)