RE: iSCSI: Flow Control

[Joshua Tseng <jtseng@NishanSystems.com>]

I'm not a s/w engineer, so I could be getting myself into trouble here,
but I still see an issue....

iSCSI will not know whether the next PDU in the TCP buffer is a command
or data PDU until it actually reads the data.  iSCSI should NEVER stop
reading
data from TCP or else deadlock if the next message is a data PDU (as was so
eloquently explained to me in a prior message).  It must continue reading
data (thus emptying the TCP receive window), and if the SCSI command queue
is full, then SCSI will respond with TASK SET FULL.  This works okay
(depending
on who you ask) in a parallel SCSI or FC world where latency is measured in
us,
but in iSCSI it could be 100ms and 1000+ commands later before the initiator
ever finds out.

Josh

-----Original Message-----
From: David Robinson [mailto:David.Robinson@EBay.Sun.COM]
Sent: Wednesday, September 20, 2000 3:51 PM
To: ips@ece.cmu.edu
Subject: RE: iSCSI: Flow Control


> As I understand it, TCP should not have any intelligence
> of the iSCSI layer above it.  Thus, TCP does not know if
> the oncoming data is an iSCSI data PDU or a command PDU.
> If it is receiving a long stream of data, TCP can't tell
> if it's 1 short command PDU and a very long data PDU, or
> 100 short command PDU's. 

I agree completely.

> The latter case could overwhelm the
> SCSI command queue, but still be within the TCP window of
> the receiving node.  

This I don't understand. The act of TCP receiving data into
its window (and thus causing the window to shrink) is
complete independant of the application (aka iSCSI) grabbing
the data (causing the window to grow). As an extreme example
a naught initiator sends 1,000 commands down to a target that
has a command queue depth of 2, will it be overwhelmed?  I think
not.  A sane implementation of the target will detect that there
is data on the connection, read a chunk large enough to grab
the first command and stuff it in the command queue, read the
next command and stuff it in the command queue, then simply
waits for one of the commands to finish leaving the other 998
commands in the TCP receive buffers.  When one of the commands
completes it sucks in the next command and puts it in the queue.

> That is, unless you want iSCSI to talk
> to TCP to tell it to close the window.  But that would
> violate layering principles, I believe.

The above example violates not layering, the TCP window closes because
the application does not read all of the data because its command
queue is too small.  Now a clever implementation can do interesting
tricks in hardware where it copies the data stream straight into
the command queue based on its interpretation of the data stream.
The overly clever (and thus dangerous) implementation could also
play with the window and not open it up until *after* the command
is executed.  But again this is an implementation hack that 
saves a copy but then violates layering, but the proposed protocol
does not violate layering.

Still don't see the issue.

	-David