Re: iSCSI: Flow Control

[csapuntz@cisco.com]

> As was discussed earlier, TCP may need to buffer 16MB or more of
> data in order to work at gigabit speeds and 100+ms of latency.  With
> this much data being buffered in the initiator's transmit buffer,
> how does this affect the initiator's ability to send task management
> functions?  

The transmit buffer, I think, is an artifact of current designs
rather than a fundamental feature of TCP. TCP only requires a 
retransmit buffer;  I don't think it needs a transmit buffer.

A suggestion from Randy Haagens and some others: The initiator could
treat the TCP connection as a link and schedule iSCSI PDUs on that
link as room becomes available on the link.  If a task management
command appears in the initiator queue, it could be scheduled first to
go out on the link when there is room.

To allow maximum availability of the link, the iSCSI PDUs sent should
never exceed the capacity available, i.e. the remote window. This
means, don't send 1 gigabyte data PDU into a 64k remote window.

Then, the secret then is to keep the TCP stream smoothly flowing at
the target. CmdRNs keeps the TCP connection from stalling due to
excess commands. The ability to drop immediate data can be used
to keep the connection flowing at the target.

Because of the FIFO nature of TCP, of course, you'll have to wait on
everything that was sent prior to the task management command
to be received. This might take a while, esp. in the presence of
retransmits. But I think it's a small price to pay for the 
conceptual simplicity of the TCP virtual circuit.

No doubt when we go to SCTP, there will be special modes where we
can send task management commands out-of-band.

-Costa