RE: iSCSI: Flow Control

["Douglas Otis" <dotis@sanlight.net>]

Pierre,
<snip>
> Let me answer to your objections to a) and d) here.
> First d)
> Let's assume we have a iSCSI adapter of the type SCSI/FC on which
> the server posts commands and is only informed of the completion.
> Every thing else is managed by the adapter.
> In fact this adapter works as a pull machine.
> It contemplates a bunch of commands posted (with pointers
> on the data buffers associated) and decides what to pull
> from host memory after each iSCSI PDU sent on the wire.

This is describing FC as it exists with use of the command window but rather
than frame control, this has been changed to command control.  This lacks
the benefit of improving write latency with unsolicited WRTCMD-DATA.  To
allow adequate resolution, going back to a frame control would once again
restore the required control and allow mixed commands and data.

If you exclude WRTCMD-DATA, then the adapter would have no choice but to
send commands unsolicited.

> Here you can add cleverness in the adapter to help us:
> 1  after each PDU is sent the adapter enter a loop:
>
> 2  for each command posted, encapsulate and
>    send the command (in order). Do that for all the commands
>    posted to it and not yet sent (stop if the command window is closed)
> 3 when the commands requests are exhausted
>    the target sends data till a new command is posted. At
>    this point it restarts in 1
>
> Doing that allows the commands to pass the data.
> However the order between the commands is preserved.
>
> Now if you use another kind of adapter (for example
> a regular LAN adapter and if you run TCP/IP
> on the host, it will not work.

This statement is not altogether accurate.  You are assuming data requests
by the target are sitting on a transmit queue. If data is requested in
smaller bursts and handled in a similar fashion to TCP windowing, then you
would not jam the transmit buffer.  This would require an intelligent target
that knows not to get too far ahead of data requests.

> However, the main market will be using adapters
> "a la" SCSI, no?

No. If you require separate network connections to a client, then you have
substantially reduced the benefit of making the SAN common to IP.  FC
already works well and would make a better choice if two adapters would be
required.

> About the objection in a), has commands can pass data
> i don't see a problem (as far as unsolicited data has
> been authorized (no R2T required)). You can post
> a huge WRITE to the adapter let say (1MBytes).
> The adapter will slice it in iSCSI PDUs and will be
> able to interleave other commands.

The slicing would need to happen before sending to the adapter interface.
Once at that level, it is out of the control of the SCSI driver.  As threads
typically attempt to get their job done as quickly as possible, a pacing
algorithm would be required to choke data just below the network rate to
allow command leeway.

> It is why it seems to me that the model we have in the draft
> combined with adapters as described above is a very good
> model in term of performance, even with
> just one TCP connection.

The real effort is to not change TCP to suit SCSI.  Once you get the
command/data somewhat under control, the next effort will be controlling the
FIFOs feeding the medium.

Doug