Re: iSCSI/iWARP drafts and flow control

[Mike Ko <mako@almaden.ibm.com>]

Caitlin, it looks like as far as iSCSI asynchronous messages are 
concerned, we both agree that "setting limits on most of the types listed 
is so easy that there is no need to negotiate the limit".  So the only 
other item is the SCSI asynchronous events, and the question yet to be 
answered is how many AENs can realistically be sent at one time.  Since 
the optional Shared Receive Queue already provides a solution to handling 
this situation without impacting the iSCSI spec, does anyone else thinks 
that the handling of AENs justify adding a negotiable item to set a hard 
limit on the number of AENs?

Mike Ko
IBM Almaden Research
San Jose, CA 95120

Sent by:        owner-ips@ece.cmu.edu
To:     Mike Ko <mako@almaden.ibm.com>
cc:     ips@ece.cmu.edu 
Subject:        Re: iSCSI/iWARP drafts and flow control




On Sunday, July 27, 2003, at 02:21 PM, Mike Ko wrote:

> From sec.3.5.2.1 of the iSCSI spec, "Asychronoous Messages are used to
> carry SCSI asyncrhonous events (AEN) and iSCSI asynchronous messages."
> Sec.10.9.1 lists the codes for Asyncrhonous Messages which include 0)
> SCSI
> Asynchronous Event, 1) target requests Logout, 2) target indicates it
> will
> drop the connection, 3) target indicates it will drop all connections
> of
> this session, and 4) target requests parameter negotiation.  Since the
> iSER layer can estimate the number of Asynchronous Messages needed in a
> typical situation for the AsyncEvent codes listed in the iSCSI spec, it
> can determine how much to overprovision when a dedicated receive queue
> is
> used.  Additionally, if one is concerned about the atypical case, the
> optional features of graceful handling and the shared receive queue as
> you
> have pointed out can be used.  So it doesn't seem to me that flow
> control
> for Asynchronous Messages are needed.  Can you provide an example
> where an
> unbounded number of Asynchronous Messages would need to be sent?

If it is so easy to bound the number of Asynchronous Messages without
specific knowledge of your peer, why not simply state that limit in
the iSER spec?

If you cannot agree on that limit for a spec, how are two peers supposed
to do so in the wild?

iWARP ULPs are required to take responsibility for flow control That
means
the Data Sink must know how many untagged messages are allowed to have
in flight. Period. It does not mean that there has to be that many
buffers pre-committed.  Just that there is an agreed upon limit.

There is already an agreed-upon limit for 99.99% of the traffic,
untagged
messages that have CmdSNs. It is almost silly to reduce the protocol
from
being truly reliable to being statistically reliable for exceptions.

Setting limits on most of the types listed is so easy that there is no
need to negotiate the limit. It would be absurd to have two "I will drop
this connection" messages in flight on the same reliable connection.

That leaves the Asynchronous Events. As that these are dependent on the
type of target it would seem to be a bit more difficult to meaningfully
state a maximum number of in-flight messages that a future device might
need to generate. So you have to include an "in-flight" limit during
setup,
and then you need to a way to know when the message is no longer
"in-flight".
The obvious solution is a response to a *later* command.