Re: iSCSI/iWARP drafts and flow control

[Mike Ko <mako@almaden.ibm.com>]

Caitlin, since MaxCmdSN is used by the target to limit the number of 
commands it can receive from the initiator, your suggestion would work for 
limiting the number of "fringe" messages the initiator can send to the 
target.  However, for "fringe" messages from the target to the initiator, 
such as the Aysnchronous Message, the target cannot simply raise MaxCmdSN 
and expect the initiator to increase the number of receive buffers for 
"fringe" messages correspondingly.

Mallikarjun is correct in pointing out that some form of new wire protocol 
for flow control is required if we need to flow control the messages not 
regulated by CmdSN.

Mike Ko
IBM Almaden Research
mako@almaden.ibm.com

Sent by:        owner-ips@ece.cmu.edu
To:     "Mallikarjun C." <cbm@rose.hp.com>
cc:     <ips@ece.cmu.edu>, <rddp@ietf.org> 
Subject:        Re: [rddp] Re: iSCSI/iWARP drafts and flow control




On Thursday, July 31, 2003, at 07:43 PM, Mallikarjun C. wrote:

>> No new wire protocol is required.
>
> Please explain to me how credit can be replenished
> in the following example - without new positive flow
> control protocol.

I thought of at least one algorithm overnight. This is
not presented as a definitive proposal, merely to show
a strategy that can allow iSER to provide flow control
of *all* untagged messages.


iSER untagged messages are divided into a predictable
portion regulated by CmdSNs in a very direct fashion
and asynchronous messages. The latter category can be
characterized as having a low sustained rate compared
to the CmdSN-related untagged messages, but that the
traffic is very bursty. That is, the peak rate can be
high.

That suggests adapting a classic "leaky bucket" credit
system of the type typically used to regulate burst
traffic over rate controlled networks. The key
difference is that the "clock" is the Max CmdSN.

What is required is the following:

The sender maintains a asynch-credit counter. It is
initialized to a known value. This value could be
negotiated if it is believed that there is enough
variation to warrant negotiation, otherwise it would
be fixed. That caveat applies to all other "constants"
cited in this algorithm.

When the ULP desires to send an untagged message using
one of these credits:

The credit count is brought up to date: The
current Max CmdSN is compared with the value when
the prior "fringe" send was performed. The delta
is used to grant new credits, however there is a
maximum number of credits that may be accumulated.


If there are enough credits: the credit count is
reduced and the untagged message may be sent.

If there are not enough credits: the untagged
message must be delayed until the Max CmdSN is
advanced. After a configurable delay, iSER
SHOULD send some form of NOP command to cause
Max CmdSN to advance.


Note that there are NO iSER defined messages for
the purpose of flow control.  Untagged messages
may be delayed, but that could happen over TCP
just as easily. The only change is that *only*
untagged messages will be delayed, never tagged
messages, and *where* they are delayed (iSER
versus TCP).