RE: iSCSI: Flow Control

["Douglas Otis" <dotis@sanlight.net>]

Charles,

The problem with the iSCSI control schemes is that it does not relate to the
actual FIFO being controlled.  A dynamic credit scheme as I indicated in the
example draft would allow an accurate tracking of buffer space and allow
adjustment to changes in use to be tracked rather quickly.  The internal
FIFO feeding the medium is the critical parameter being controlled as you
would not want one FIFO to become deep.  Otherwise this would allow frames
to become stale.  It would be better to keep these frames at the client in
the case where the FIFO feeding the medium is congested.  For good
operation, all these FIFOs must be kept as lean as possible.  That would
imply Class 3 control.  This type of control is already in use, so you might
say the iSCSI scheme has not been tested.

The present iSCSI scheme makes no provisions for such control.  There are
not any meaningful constraints in adhering to the limits imposed by FCP in
limiting to 256 commands.  For a 125 mile MAN distance at 10G-bit, FC would
be limited to 128K-sequences per second with 500 responses per second which
implies 256 concurrent threads would be required to satiate.  Access to just
two devices could conceivably maximize the connection bandwidth at 8k-bytes
per sequence.

Doug

<snip>
> > B) We need of course a flow control on the data
> > because even if we have a flow control on the commands,
> > just a few commands with huge unsolicited data could
> > overwhelm the target buffers.
> >
>
> I agree with John and others on this point. The problem with a
> strict credit
> scheme is resource underutilization. For that reason, I like
> something more
> like Randy Haagen's proposal in an earlier posting, where the logical unit
> advertises static advisory values that work "most of the time", with the
> proviso that the initiator may still get an occassional queue full
> condition. With long flight times, of course, this would need to be
> buttressed by some method of flushing the pipeline when such an error
> occurred. That said, additional tools would still be needed so
> the initiator
> can control the amount of inflight commands and data.
>
> With regard to the various dynamic credit allocation proposals,
> I'm hesitant
> to jump on that bandwagon without some behavioral data from real life. Of
> course, there's still the ultimate feedback loop from the user to the
> storage provider when performance goes down the tubes or cost goes through
> the roof.
>
>
> > C) We need a flow control on the commands to avoid
> > the TASK SET FULL condition. If the target hit this condition
> > it has to drop command (return task set full) and
> > drops data associated. Avoiding the task set full condition
> > avoid too the "dead locks".
> >
>
> As noted above, a better approach is one that avoids queue full
> most of the
> time with some way to resychronize both ends of the pipe when it
> does occur.
>
> <remainder deleted>
>
> Charles
>