RE: iSCSI: Flow Control

["Binford, Charles" <cbinford@lsil.com>]

Title: RE: iSCSI: Flow Control

I do not understand the question of "Even if the SCSI layer uses RTT, what if ISCSI did not?  What would be the result?", so I may be also misunderstanding the general issue.  However, let me add my two cents to what I think I'm hearing.

It appears to me that David Robinson is suggesting that there is no need for targets to ask for the data (RTT), the initiator should just send it immediately after the CDB.  "Modern" RAID controllers with large amounts of memory can buffer it. 

I disagree with that line of thinking.  SCSI has always had the target in control of when the data flowed, and for good reason.  Regardless of the amount of cache the target has, the host can and *will* fill it up (disks are still slower than the transport).  It is imperative for the target to be able to manage his own cache, and that includes being in control of timing of the data transfer.  By having the target specify when the data is transferred you leave the data in the host until the target has space for it.  If not, the data ends up in the transport layer with only flow control to slow it down, causing unneeded congestion.

FC allows for the option of having data sent immediately after the CDB.  However, there is a strict limit on the amount of data that can be delivered in that initial chunk (set via a mode page).  If the Write is larger that what is allowed by the initial burst size, then the target uses XFER_RDY to ask for the remainder of the data.  This type of scheme is acceptable because a target can set aside a chunk of data memory for each queue slot it has and use the XFER_RDY to manage the receipt of any data for that I/O that didn't fit in the pre-allocated buffer.  (Of course you still have the queue-full management problem, but that is a separate problem than what I'm addressing here.)

Charles Binford
LSI Logic Storage Systems
(316) 636-8566

> -----Original Message-----
> From: John Hufferd/San Jose/IBM [mailto:hufferd@us.ibm.com]
> Sent: Thursday, September 21, 2000 6:32 AM
> To: Reflector, IPS
> Subject: Re: iSCSI: Flow Control
>
>
>
> Matt,
> I am not sure that you answered David's Question.  Even if
> the SCSI layer uses RTT, what if ISCSI did not?  What would be the result?
> .
> John L. Hufferd
>
>
>
> "Matt Wakeley" <matt_wakeley@agilent.com>@ece.cmu.edu on 09/21/2000
> 12:35:55 AM
> Please respond to Matt Wakeley <matt_wakeley@agilent.com>
> Sent by:  owner-ips@ece.cmu.edu
>
> To:   "Reflector, IPS" <ips@ece.cmu.edu>
> Subject:  Re: iSCSI: Flow Control
>
> Well, "modern" disk arrays imployed on SCSI and FC use
> XFER_RDYs (RTT). They don't seem to question it.
>
> -Matt
>
> David Robinson wrote:
>
> > Matt Wakeley wrote:
> >
> > > Ok, so now the SCSI processes the first command, and sends an RTT
> (XFER_RDY in FC terms) to the initiator.  Now, the initiator sends the
> data down the same TCP connection, and it gets stuck behind all those 998
> commands in the TCP receive buffers.  The command can't complete because it
> can't get the data, and the data can't be delivered because there's no room for the
> commands in front of it.  Deadlock.  Do you see the issue now?  (this
> is a good example of why the single TCP connection model, be it synchronous or
> asynchronous, is bad).
> >
> > As I said in other e-mail I question the use of RTT in a modern
> > environment with large buffering. Data immediately following the
> > command makes much better sense to me, but if we must support
> > this environment then you are right that deadlock can occur.
> >
> >         -David