RE: A Transport Protocol Without ACK (resend)

[Michael Krause <krause@cup.hp.com>]

At 12:04 PM 9/22/00 -0600, HAAGENS,RANDY (HP-Roseville,ex1) wrote:
>Robert,
>
>The reason we proposed immediate write data for iSCSI was as an optimization
>for short writes over "long" networks.  As network diameter increases,
>minimizing round-trip signalling delays becomes important.  Sending write
>data immediately following the command eliminates the need for an RTT from
>the target, and thus eliminates a round-trip delay.
>
>If immediate data threatens to overflow the target's cache, the target may,
>on an emergency basis, discard the data; and, when ready, request data again
>using the RTT protocol.  This behavior should be minimized, however, as it
>will tend to congest the network.
>
>Note the use of SCSI-layer flow control (RTT mechanism).  I agree with those
>that say a controller must manage its cache, regardless of the size of the
>cache.  This is most apparent when one considers multiple hosts sharing a
>storage controller.  In order to allocate performance between the hosts, the
>controller must ensure that a write burst from a single host cannot fill its
>cache.  RTT provides a means for doing this that still allows other commands
>(read commands, for example) from the same host to proceed.
>
>However, reserving a few MB of cache for each host, to accommodate immediate
>writes, seems to me reasonable.  The command windowing mechanism (another
>flow control mechanism) will prevent the overrunning of these immediate
>write buffers, assuming that the controller has reserved space in proportion
>to the largest immediate buffer allowed (which is negotiated) x the maximum
>number of commands.
>
>Upon writing this, it occurs to me that it may be useful to add an
>XOF/XON-like mechanism for temporarily disallowing immediate write data.
>This would give the controller another tool for cache management.  When its
>cache became congested, the controller could signal one or more of the
>requesting hosts to stop sending immediate write data with commands; later,
>another message could allow immediate writes to resume.  These messages
>would need to make it to the iSCSI layer in the hosts, as they will affect
>how the host does DMA chain construction.

Given the distances involved and the subsequent latency, I'd suggested 
lifting the technique of issuing a RNR-NAK (receiver not ready) with a time 
period the sender should wait upon receiving the NAK before it retries the 
operation (0 indicates immediate which may be acceptable if one knows the 
sender is far away).  Allows the receiver to allocate the requested buffer 
space within what it believes to be a reasonable amount of time, eliminates 
the need for the receiver to track the sender's state for this problem, and 
eliminates the additional message associated with the XOFF/XON mechanism.

In general, we should strive wherever possible to architect such that only 
one side needs to track the state of a given attribute - this leads to 
simplicity in implementation and future development of the architecture.

Mike