twist on iSCSI asymmetric model

[meth@il.ibm.com]

In the asymmetric model, one TCP connection (the control connection) is
used for sending all of the commands, thus ensuring ordered delivery of the
SCSI commands. If (a large amount of) data is also sent on the control
connection, there is the possibility that the data would fill up the
window, and critical task management requests couldn't get through.

On the other hand, it has been pointed that some implementations will
prefer a single (and only) TCP connection for both commands and data.

I would like to suggest the possibility of combining these 2 cases; either
     (1) have a single connection for everything
or
     (2) have one connection for control and one or more data connections.
In this case, no (WRITE or READ) data may be sent over the control
connection.

The initiator and target negotiate during the first login as to which of
the 2 possibilities they will use.

In (1), it is the responsibility of the initiator to worry about not
overloading the one and only connection so as not to delay delivery of
critical task management messages.

In (2), the iSCSI SCSI Command PDU specifies which data connection to use
for the data of this command.

On the data channels, where we will have iSCSI Data and R2T PDUs, we
shouldn't need to use the long headers that we use to specify an iSCSI SCSI
Command PDU. We will therefore have a shortened header for the data
channels. In (1), since data will travel on the control channel, the iSCSI
Data PDU and R2T headers will be padded with zeros so that all headers are
of the same length on a particular connection.

The use of R2T is negotiated during the first login of the iSCSI session.
If R2T is to be used, then the initiator may not send data before it is
requested by the target. Thus there will always be buffers ready to accept
the data. If the target agrees to not use R2T, then the target implicitly
agrees to have some number of buffers available to receive data sent by the
initiator, even if the data arrives before the command arrives. In this
case the initiator MUST send data in the data connections in the same order
that the corresponding commands were sent. (This avoids deadlocks.) If the
target runs out of buffers, then it will let data back up in the TCP window
of the data connections, while it is processing the data from earlier
commands.

The only main drawback remaining with the asymmetric model is that 2
separate operations may be required to set up sending a command and to
register its DMA buffers. If the control and data connections happen to be
on the same NIC, then this might not be a problem.

Only one interrupt is required upon completing a READ data transfer, if we
mark the last iSCSI Data PDU of the data transfer.

- Kalman Meth.