Re: iSCSI: Flow Control

[Julian_Satran@il.ibm.com]

Pierre,

The only point you are missing is that the TCP window may be closed when
you want to send your
Read command and even if not it will reach the other end after all the data
before it
regardless of how clever your adapter is.  The FIFO you have in mind is
certainly not
equivalent to the pipe capacity.

Julo

Pierre Labat <pierre_labat@hp.com> on 10/10/2000 02:58:41

Please respond to Pierre Labat <pierre_labat@hp.com>

To:   ips@ece.cmu.edu
cc:
Subject:  Re: iSCSI: Flow Control




Julian_Satran@il.ibm.com wrote:

> Pierre,
>
> It does not matter how from where you send the data on the wire.
> If you have a long wire and you want to cover the latency you will
> send data as soon as you can and then commands get stuck  behind.

Julian,

The command can NOT  be stuck because there is "data on the wire".
Let me give you an example,
Let's talk again about the "pull model" adapter on the initiator.
Imagine you have 100Mbytes of (write) data outstanding
because 1000 cmds of large write commands have been posted to
the adapter.
The adapter sends this data as fast as it can. But very important,
the data are not tossed in any kind of buffer on the adapter.
What the adapter does is: pull some kbytes of data form host memory,
encapsulate it, send it on the wire. Again and again, as fast as it can.

Now, imagine that a read is posted to the adapter after the 1000 writes.
Here is the point. The interface between the host and the adapter is not
a FIFO but a flat array and the adapter can works in parallel on
all the commands. Immediately when the host posts the read
(in the flat array), the adapter sees it. The adapter as soon as it
completes transmitting the current data PDU, sends the read command.

The read command is not stuck behind the 100Mbytes of data.
The maximum latency for the command is the time to
transmit one iSCSI pdu on the wire.
That is (size of pdu)/throughput.
 Then the adapter continues to send the write data of the
100Mbytes. And as soon as a new command will be posted,
it will send a command pdu immediately after the current
data PDU.

Commands are not stuck behind data because there is no FIFO
before the wire, and because data "on the wire" doesn't block anything.
The wire is always able to deliver its throughput.


Regards,

Pierre

>
>
> And nobody is suggesting you should park the data on the NIC card if
> you know better.
>
> Julo
>
> Pierre Labat <pierre_labat@hp.com> on 09/10/2000 20:41:14
>
> Please respond to Pierre Labat <pierre_labat@hp.com>
>
> To:   Julian Satran/Haifa/IBM@IBMIL
> cc:
> Subject:  Re: iSCSI: Flow Control
>
> julian_satran@il.ibm.com wrote:
>
> > Pierre,
> >
> > Sorry I missed a point about a - I though you where saying that
> unsolicited
> > data
> > are not allowed. On this we are in agreement.
> >
> > On the rest - I can hardly follow. The model you suggest while valid in
a
> > close
> > scheme like a bus or short serial connection - in which the target
> fetches
> > data is closely matched by th R2T for data with no such match for
> commands.
> > Keeping track of how many commands where shipped for what LU is
> impractical
> > as we don't what per-LU state at the initiator (for the same reason we
> > rejected
> > the connection per LU model).
> >
> > As for D - the point is that when you have a command to send and the
> > command window
> > is open you might have to wait a long time as the TCP window is closed
> > and/or you have
> > a lot of data ahead.
>
> I think there is a misunderstanding about the model i was talking about.
> It's a pull model as implemented in some FC cards today and it is assumed
> that
>
> TCP/IP is handled on the adapter. It is the "no memory on adapter" model
> Somesh talked about.
>
> When a command comes out the SCSI layer, it is posted to the adapter.
> At this point it is not posted in a queue but in a flat array of
commands.
> The data is till in host memory.
> Let's assume the card can handle 1000 commands in parallel, the array
> has 1000 entries.
> The adapter is able to process this commands the way it wants
> as far as it respects the protocol (iSCSI in our case). It could
> be able to process them all in parallel if needed.
> As it is a flat array, no commands are blocked by an other commands
> or data. The adapter can pick (pull) whatever command or data
> from host memory and send
> it on the wire (again as far as it respect the protocol).
>
> Regards,
>
> Pierre