Re: iSCSI Autosense Consensus, Connection next steps

[csapuntz@cisco.com]

Asymmetric connections may be the way to go.

Let's look at characteristics of control vs. data:

* iSCSI control
	- low bandwidth (handled on a CPU)
	- should generally be processed in FIFO order
        - complex state machines (probably software implemented)
        - flow control nice
	- doesn't need to end up in any special buffers
	  (can be processed in-place)

* iSCSI data/ready to transmit (rtt)
        - potentially high bandwidth 
	- with appropriate headers, segments can be 
          processed entirely out-of-order
        - simple data transfer state machine
        - no flow control on data needed, just congestion control
	           - rtt's need to be flow-controlled, but
                     that can be done with a simple credit mechanism
	- data needs to end up in special buffers (e.g. buffer cache)

The processing requirements of the data and control seem quite
asymmetric. Mixing them on one TCP channel makes separating out the
data and control more difficult.

Conversely, by putting control and data on separate TCP connections,
you can use the flow label (IP addresses, TCP ports) to determine
whether to route this along the control or data path in your machine.

So, I think there is value in the proposal that we use 1 control
connection and up to n separate dedicated data/RTT connections. 

The symmetric, connection allegiance case was developed when we were
smoking some really good stuff in Haifa. It really addresses a 
corner case: minimizing the communication needs between NICs in systems
that will stripe requests across NICs.

However, somewhere along the line, we forgot to optimize for the
single NIC case.

------------------

Also using a separate TCP connections may allow us to migrate one
day to an architecture where a different node in the IP network
returns the actual data from a SCSI READ!

Client <---- iSCSI Control ----> Storage Controller <---- TBD ----> Disk array
  ^                                                                    ^
  +-------------------------------iSCSI Data protocol -----------------+


-Costa