RE: Performance of iSCSI, FCIP and iFCP

["Somesh Gupta" <somesh_gupta@worldnet.att.net>]

Victor,

I just assumed that you were familiar with the iSCSI specification
of multiple connections per session - which is essentially as you
describe in the second last paragraph - there is one big "flow"
between the initiator and the target (called an iSCSI session)
which is then striped across multiple TCP connections - and I
think people assumed that your statements were in support of
such a proposal.

Having said that, the math still holds. And also the fact that
fundamental limits to the throughput of a single TCP connection
really do require a change to TCP congestion avoidance (esp when
mistaking link error for congestion) formulae.

If the link error rate is such that errors occur faster than time
to recover (or of that order), TCP will have to accomodate that
somehow. I don't think the TCP gurus are inflexible in this area
- PAWS, window scaling, and SACK are all example of TCP accomodating
the characteristics of the underlying network.

Somesh



-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Victor Firoiu
Sent: Tuesday, January 16, 2001 7:54 AM
To: somesh_gupta
Cc: Stephen Bailey; ips; Franco Travostino
Subject: Re: Performance of iSCSI, FCIP and iFCP


Somesh,

my answers between your lines.

Victor

Somesh Gupta wrote:
> 
> Victor,
> 
> First I would like to address the second case where - based on the
> need to be fair - you raise the point that multiple connections per
> session is closer to emulating multiple users on a server than a
> single connection. That being a valuable point, is probably better
> addressed at the upper layers of the host software, where multiple
> independent sessions can be created to the same target. The argument
> made more often in favor of multiple connections is that the likelyhood
> of some connections escaping RED - or other disasters - when you have
> multiple connections is good and therefore the overall performance
> will be better.
> 

Not sure what you mean by "multiple connections per session."
Maybe I was not clear, I was arguing for one TCP connection for one
storage connection, which can give multiple TCP connectons for one pair
of IP storage gateways.  The performance considerations that I presented
in my first message regard any possible packet loss, including RED.


> However, as Steph pointed out, that skirts the issue of fairness.
> 
> My point about the modifications to congestion control algorithm was
> to point out that your work makes an interesting observation(or my
> paraphrasing of it) - that it becomes less and less possible to
> acheive the maximum possible throughput (in LFNs) as the rate and
> distances get larger and larger - and that issue should be addressed
> directly. This is true even when a single "data flow" or "application"
> might have the entire TCP bandwidth at its disposal. A modification
> hopefully will not create super TCPs i.e. would be a reasonable approx
> of current equation (flow slow long links or fast short links), but
> would provide a better choice (which should propagate to all TCPs)
> for 10G and beyond at long distances.
> 

As I was saying in some other message, modifying TCP is very problematic
for many reasons, and it is definetly outside the scope of this WG.

> There is a cost of synchronizing across multiple connections - both
> on the target and the initiator. Someone has to do the work, be it
> the host or the adapter. On the initiator side, multiple
> connections/adapters (actually their completion routine) will be updating
> a single variable MaxCmdRn and the software posting the commands will
> have to ensure that MaxCmdRn is not exceeded and has to post the commands
> to the appropriate queues (how does it know which). And that is when
> there is no stall. Assuming this informations is written/read from
> multiple processors (with the pinging of the cache lines), the cost
> will escalate rapidly with the number of processors involved.
> 
> There is similar situation on the target. It probably
> will prevent different controller boards on the target from sharing
> connections in a session (whenever one of the controller decides on a
> new increased value of MaxCmdRn, it should be propagated to the other
> controller also ??) - maybe this is not so.
> 
> That is why I am very interested in any pointers you or anyone else
> could be provide on high-speed implementation of a "single data-flow"
> across multiple connections. I am sure this has been tried in the
> past so hopefully there are some pointers to results.
> 
> Somesh
> 

I think that what you are saying is somewhat orthogonal to my points.  I
was arguing for one TCP connection for one storage connection, which is
different, simpler and more natural than bundling or anonymizing all
storage connections in one big flow and then striping a "single
data-flow" across multiple connections.  

I am not sure about other literature, I'll be looking for it myself.

Victor