RE: iSCSI: Flow Control

[Jim McGrath <Jim.McGrath@quantum.com>]

Yes, C1, C2, D1 is very common and is required.  The reason is that the
target can often dramatically improve throughput (different from latency) by
executing commands out of order.  So you want to give the target ALL of your
queued commands (or up to a reasonable number, like 256) and let it drive
the execution.  Note that even the initial latency is quickly hidden if you
really do have a lot of commands to execute.

However, this is still not a problem as long as the window is big enough
(i.e. more than the credits you have outstanding), and you took into account
the buffer space needed for commands as well as data (as per a previous
email).  Since the target window and credits should reflect the same buffer
constraints, I'm thinking that there may be a problem if the maximum window
and the maximum credits are changeable at different times (i.e. it does not
good to realize on the fly that your underutilized buffer can supports a lot
more credits than you initially thought if changing the maximum window size
takes too long to take advantage of these resources).

Any thoughts as to whether this is an actual problem?  (I am thinking of
maximum changes occurring on the minute level time scale).

Jim


-----Original Message-----
From: David Robinson [mailto:David.Robinson@EBay.Sun.COM]
Sent: Wednesday, September 20, 2000 5:13 PM
To: ips@ece.cmu.edu
Subject: RE: iSCSI: Flow Control


> I'm not a s/w engineer, so I could be getting myself into trouble here,
> but I still see an issue....
> 
> iSCSI will not know whether the next PDU in the TCP buffer is a command
> or data PDU until it actually reads the data.  iSCSI should NEVER stop
> reading
> data from TCP or else deadlock if the next message is a data PDU (as was
so
> eloquently explained to me in a prior message).  It must continue reading
> data (thus emptying the TCP receive window), and if the SCSI command queue
> is full, then SCSI will respond with TASK SET FULL.  This works okay
> (depending
> on who you ask) in a parallel SCSI or FC world where latency is measured
in
> us,
> but in iSCSI it could be 100ms and 1000+ commands later before the
initiator
> ever finds out.

The TCP connection can have either a command or data.  In either
case reading a short fixed amount of data can determine if it is
one or the other. With a single connection I can only see deadlock
occuring if the initiator sends Command1, Command2, then Data1 on a single
command/data connection. With a single connection this seems like
a broken initiator as doing the right thing of C1, D1, C2 shouldn't
be any more expensive and it is definately safer. Well one other
case can occur is if the initiator wants the target to issue an RTT
and it interposes another command before the RTT is received, this
too seems like a broken things to do as I don't see much value in doing RTT
on a single connection.

In the multiple connection asymettric case this isn't an issue as data
is never interleved with commands.

So is there a fundamental requirement that the target must handle
the case of C1, C2, then D1? It seems easy to just write the iSCSI
initiator to never issue such a sequence and if the target gets
one return an error when C2 arrives before D1.

It again seems like we are trying to impose or solve datagram problems
in a stream world. Does parallel SCSI have this problem?

	-David