RE: Towards Consensus on TCP Connections

[julian_satran@il.ibm.com]

We had this in draft 00! (Adelaide). It was deemed too complex by the team
and had the perceived
disadvantage of having to setup too much address-association information in
the adapters (as
every adapter has to be ready for any piece of data).

A variant of it (easier palatable) was recently suggested by Kalman Meth
(Kalman could you please
re-post it?). Kalman's proposal could be extended to enable complete "data
striping".

If the teams shows enough interested we might attempt to work it through
e-mail (a memo describing some details). It does not require more RDMA
support than the current one.

One side-effect of it is that control information runs on a single
connection and command sequencing becomes unnecessary and flow control is a
TCP window (I think).

Julo


Stephen Byan <Stephen.Byan@quantum.com> on 10/08/2000 16:04:27

Please respond to Stephen Byan <Stephen.Byan@quantum.com>

To:   "'ips@ece.cmu.edu'" <ips@ece.cmu.edu>
cc:    (bcc: Julian Satran/Haifa/IBM)
Subject:  RE: Towards Consensus on TCP Connections




David Black wrote:

> (B) Should iSCSI have a session abstraction that
>    binds multiple TCP connections into one
>    iSCSI connection?
[snip]
> Let me start from Costa's simple explanation:
>
> > There are two ways in which multiple connections are used in iSCSI:
> >
> > 1) Multiple simultaneous TCP connections for bandwidth
> >
> > 2) Multiple TCP connections for fault tolerance/recovery
> >  (i.e. when one TCP connection in a session dies,
> >          another one starts up)
>
[snip]
> So, to resolve issue (B), we need to discuss whether
> there are compelling reasons for doing 1) and/or 2)
> within a single SCSI connection, as opposed to across
> multiple SCSI connections.  This specific issue needs
> further discussion -- I invite everyone to have at it ...

As I've explained in a previous post, I see a need for low-cost bandwidth
in
excess of 1 Gb/s for low-end single-LUN iSCSI disks. This bandwidth should
be available to a single iSCSI command from an initiator.

I believe the only low-cost link technology available to us is 1 Gb/s
ethernet over cat5 UTP (is this 1000baseT in IEEE-speak?). To get
sufficient
bandwidth at sufficiently low cost in early device-level implementations, I
think the use of multiple TCP connections over 1 Gb/s ethernet links is the
only feasible option. 10 Gb/s ethernet will be too expensive for the
initial
device-level implementations, especially since I think device-level
customers will require redundant interfaces, so a device would have to bear
the cost burden of two 10 Gb/s interfaces.

So I think the requirements for multiple connections go beyond what is
currently in draft-satran-iscsi-01.txt, which only allows load-sharing at
the SCSI command-level. Note that for disks, load-sharing at the
command-level could be obtained by multiple SCSI connections rather than
multiple TCP connections. I think it is important to increase the bandwidth
available to the data transfer for a single SCSI command, so an extension
to
the current multiple TCP connection proposal looks like the right answer.

The extension I propose is to remove the allegiance to the initiating TCP
connection for data transfers _only_. Not RTT's, not commands, not status,
not task management function requests, etc. Only data transfers. Note that
this probably implies something like the iSCSI over VI over TCP proposal,
in
order to decouple the adapter state needed for RDMA data transfer from the
command state.


When I consider tapes, the alternative of multiple SCSI connections looks
even less appealing. Unlike disks, tape reads and writes are not
idempotent,
so SCSI-level error recovery is quite complex because the initiator needs
to
discover the current state of the tape. Multiple SCSI sessions don't
directly address this problem; all they offer are redundant paths to the
device. The iSCSI session concept looks good because it offers a way to
retry individual information units and continue the execution of a SCSI
command in the presence of link errors, without having to invoke SCSI-level
tape error recovery.

With respect to tape device bandwidth requirements, I don't have an
informed
opinion. However, I note that if the device-level requirement should exceed
the speed of economically-feasible links, multiple SCSI connections do not
solve the bandwidth aggregation problem, since the bandwidth must be
applied
to the data transfers for an ordered set of SCSI commands. Multiple SCSI
connections do not preserve the ordering of commands. So if the tape device
bandwidth requirements exceed that of a single link, multiple TCP
connections would be required.

Regards,
-Steve

Steve Byan
<stephen.byan@quantum.com>
Design Engineer
MS 1-3/E23
333 South Street
Shrewsbury, MA 01545
(508)770-3414
fax: (508)770-2604