RE: Towards Consensus on TCP Connections

[Stephen Byan <Stephen.Byan@quantum.com>]

David Black wrote:

> (B) Should iSCSI have a session abstraction that
> 	binds multiple TCP connections into one
> 	iSCSI connection? 
[snip]
> Let me start from Costa's simple explanation:
> 
> > There are two ways in which multiple connections are used in iSCSI:
> > 
> > 1) Multiple simultaneous TCP connections for bandwidth
> > 	
> > 2) Multiple TCP connections for fault tolerance/recovery
> > 	(i.e. when one TCP connection in a session dies, 
> >          another one starts up)
> 
[snip]
> So, to resolve issue (B), we need to discuss whether
> there are compelling reasons for doing 1) and/or 2)
> within a single SCSI connection, as opposed to across
> multiple SCSI connections.  This specific issue needs
> further discussion -- I invite everyone to have at it ...

As I've explained in a previous post, I see a need for low-cost bandwidth in
excess of 1 Gb/s for low-end single-LUN iSCSI disks. This bandwidth should
be available to a single iSCSI command from an initiator. 

I believe the only low-cost link technology available to us is 1 Gb/s
ethernet over cat5 UTP (is this 1000baseT in IEEE-speak?). To get sufficient
bandwidth at sufficiently low cost in early device-level implementations, I
think the use of multiple TCP connections over 1 Gb/s ethernet links is the
only feasible option. 10 Gb/s ethernet will be too expensive for the initial
device-level implementations, especially since I think device-level
customers will require redundant interfaces, so a device would have to bear
the cost burden of two 10 Gb/s interfaces.

So I think the requirements for multiple connections go beyond what is
currently in draft-satran-iscsi-01.txt, which only allows load-sharing at
the SCSI command-level. Note that for disks, load-sharing at the
command-level could be obtained by multiple SCSI connections rather than
multiple TCP connections. I think it is important to increase the bandwidth
available to the data transfer for a single SCSI command, so an extension to
the current multiple TCP connection proposal looks like the right answer.

The extension I propose is to remove the allegiance to the initiating TCP
connection for data transfers _only_. Not RTT's, not commands, not status,
not task management function requests, etc. Only data transfers. Note that
this probably implies something like the iSCSI over VI over TCP proposal, in
order to decouple the adapter state needed for RDMA data transfer from the
command state.


When I consider tapes, the alternative of multiple SCSI connections looks
even less appealing. Unlike disks, tape reads and writes are not idempotent,
so SCSI-level error recovery is quite complex because the initiator needs to
discover the current state of the tape. Multiple SCSI sessions don't
directly address this problem; all they offer are redundant paths to the
device. The iSCSI session concept looks good because it offers a way to
retry individual information units and continue the execution of a SCSI
command in the presence of link errors, without having to invoke SCSI-level
tape error recovery.

With respect to tape device bandwidth requirements, I don't have an informed
opinion. However, I note that if the device-level requirement should exceed
the speed of economically-feasible links, multiple SCSI connections do not
solve the bandwidth aggregation problem, since the bandwidth must be applied
to the data transfers for an ordered set of SCSI commands. Multiple SCSI
connections do not preserve the ordering of commands. So if the tape device
bandwidth requirements exceed that of a single link, multiple TCP
connections would be required.

Regards,
-Steve

Steve Byan
<stephen.byan@quantum.com>
Design Engineer
MS 1-3/E23
333 South Street
Shrewsbury, MA 01545
(508)770-3414
fax: (508)770-2604