Re: twist on iSCSI asymmetric model

[meth@il.ibm.com]

John,

I'm not really saying anything new. I'm essentially summing up and choosing
the best of the various ideas that were thrown around for the so-called
asymmetirc model. The point is to not allow immediate data in the control
channel when using the asymmetric model; all bulk data must go in the data
channels. This avoids the problem of data clogging up the control channel
and thereby preventing a critical task management operation from getting
through. Since there are also strong feelings that we should allow a single
TCP connection, I prescribed how we should deal with the 2 cases: either a
single (and only) TCP connection under the sole responsibility of the
initiator to not become clogged, or a purely asymmetric model with no bulk
data going over the control channel. I then described a few technical
details that I think will allow for more efficient implementations.

The last sentence is a repeat of my response to Pierre's objection about
requiring 2 interrupts upon the completion of a READ operation: one
interrupt on the NIC receiving the data and one interrupt on the control
channel NIC for the response to the command. I am simply commenting that we
only need the interrupt on the data channel when the last data PDU arrives,
while no interrupt is needed on the control channel NIC.

- Kalman Meth



"John Hufferd/San Jose/IBM" <hufferd@us.ibm.com> on 28/09/2000 22:31:56

Please respond to "John Hufferd/San Jose/IBM" <hufferd@us.ibm.com>

To:   ips@ece.cmu.edu
cc:    (bcc: Kalman Meth/Haifa/IBM)
Subject:  Re: twist on iSCSI asymmetric model





Kalman,
I am not sure what your new suggestion solves. Especially if what Charles
wrote is correct:
{Snip}
"2.  As I recall (possibly not very accurately) SAM-xx states that an
initiator should not have more than one pending task management request at
a
time.  In general, such requests are "think-time" limited and therefore non
blocking, so this seems not to be a problem in practice.

3. It's important that this set of control functions flow over the same
control connection that's used for commands (ie.  these functions need to
flow through the command delivery pipe).  Otherwise their behavior is
indeterminate.  An example is an "abort task" function which arrives at the
target while the command to be aborted is still in transit.

4.  Considering the rule of allowing only one pending task management
request at a time,  it might be sufficient to have the initiator budget one
"credit" to be used for this purpose."
{Snip}

(His use of Credit here meant a reserved Buffer)

I believe that you also have determined that Two connections per Session
Solves the key problems (especially if what Charles said is correct,) and
they are on the same NIC, since you said ".... If the control and data
connections happen to be on the same NIC, then this might not be a
problem".  This, I believe, is the point that Matt has been trying to make.
He felt that two connections per session were key and should be required.

So lets put it all together; Control should flow on the same connection as
Commands, and Data on a separate connection (within the same session). This
solves the blockage issue, and both connections within the same NIC solve
the interrupt problem.  Plus it does not have iSCSI sliding windows etc.
etc.,  the implementation is relatively easy, compatible with the concept
of iSCSI HW on the NIC, and the  Wedge Drivers still work as they do today.
Based on that, I think Matt has a strong point.  What do you think?

You also added (on your last Paragraph) a statement that I think takes us
in a different direction "...Only one interrupt is required upon completing
a READ data transfer, if we mark the last iSCSI Data PDU of the data
transfer".
I am not sure what you are attempting to say here, are you making a new
recommendation?

In any event, please address the previous questions, before you amplify
your last paragraph.


.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSD San Jose Ca
(408) 256-0403, Tie: 276-0403
Internet address: hufferd@us.ibm.com
Notes address: John Hufferd/San Jose/IBM @ IBMUS
VM address: hufferd at IBMUSM54


meth@il.ibm.com@ece.cmu.edu on 09/28/2000 06:30:58 AM

Sent by:  owner-ips@ece.cmu.edu


To:   ips@ece.cmu.edu
cc:
Subject:  twist on iSCSI asymmetric model






In the asymmetric model, one TCP connection (the control connection) is
used for sending all of the commands, thus ensuring ordered delivery of the
SCSI commands. If (a large amount of) data is also sent on the control
connection, there is the possibility that the data would fill up the
window, and critical task management requests couldn't get through.

On the other hand, it has been pointed that some implementations will
prefer a single (and only) TCP connection for both commands and data.

I would like to suggest the possibility of combining these 2 cases; either
     (1) have a single connection for everything
or
     (2) have one connection for control and one or more data connections.
In this case, no (WRITE or READ) data may be sent over the control
connection.

The initiator and target negotiate during the first login as to which of
the 2 possibilities they will use.

In (1), it is the responsibility of the initiator to worry about not
overloading the one and only connection so as not to delay delivery of
critical task management messages.

In (2), the iSCSI SCSI Command PDU specifies which data connection to use
for the data of this command.

On the data channels, where we will have iSCSI Data and R2T PDUs, we
shouldn't need to use the long headers that we use to specify an iSCSI SCSI
Command PDU. We will therefore have a shortened header for the data
channels. In (1), since data will travel on the control channel, the iSCSI
Data PDU and R2T headers will be padded with zeros so that all headers are
of the same length on a particular connection.

The use of R2T is negotiated during the first login of the iSCSI session.
If R2T is to be used, then the initiator may not send data before it is
requested by the target. Thus there will always be buffers ready to accept
the data. If the target agrees to not use R2T, then the target implicitly
agrees to have some number of buffers available to receive data sent by the
initiator, even if the data arrives before the command arrives. In this
case the initiator MUST send data in the data connections in the same order
that the corresponding commands were sent. (This avoids deadlocks.) If the
target runs out of buffers, then it will let data back up in the TCP window
of the data connections, while it is processing the data from earlier
commands.

The only main drawback remaining with the asymmetric model is that 2
separate operations may be required to set up sending a command and to
register its DMA buffers. If the control and data connections happen to be
on the same NIC, then this might not be a problem.

Only one interrupt is required upon completing a READ data transfer, if we
mark the last iSCSI Data PDU of the data transfer.

- Kalman Meth.