Notes of 06/21 meeting

[Costa Sapuntzakis <csapuntz@cisco.com>]

Action items:
Action item: get rid of command reference number in SCSI response
Action item: get rid of immediate data in SCSI response

Unresolved question: Should we allow multiple RTTs to be pending
simultaneously? If so, how many?

--------------



The day started of with a discussion of SCSI error recovery.

The main motivation behind iSCSI layer error recovery was
tape drives. Many tape backup applications cannot recover
from aborted commands.

It was pointed out that the command reference number
as spec'ed was not long-lived enough to provide error
recovery. However, the task tag could be used to do
error recovery.

--------------

There was then a discussion about whether the command
reference number should be per LU or per session.

The per-LU advocates pointed out that there are no
inter-LU ordering constraints in SCSI SAM. Per-LU
provides greater parallelism and less head-of-line
blocking by providing more lines.

The per-session folks pointed out that LUs are not
an iSCSI layer concept. What's more, adding command
reference numbers per LU would necessitate adding
state per LU into the iSCSI drivers at both ends.
Unfortunately, there was no way of bounding or
reclaiming this state.

----------------

There was a lot of talk about whether we want
to support multiple TCP connections/session.

John Hufferd pointed out that SCSI load balancers already exist
that take advantage of multiple sessions (multiple SCSI busses) 
to stripe commands to a target. He argued that multiple
TCP connections are unnecessary. He also argued that no applications
make effective use of SCSI ORDERED attribute, because the
interface are not there. 

However, they have to stop and wait for ordered commands.
One application where stop and wait hurts is tape (where
all writes are ordered), so some tape applications write
self-describing blocks to tape which can be written in any order.

Remote asynchronous mirroring can also be done with ordered
writes. Hufferd argued that remote asynchronous mirroring must
be solved at a higher layer and is being solved today.

Most of those arguing for multiple TCP connection said that
    - it isn't that hard
    - it would make iSCSI better than other SCSI transports
    - it would make high-perf apps easier to write

----------------

Can a single initiator have multiple sessions? The answer
was yes.

An initiator is named by the initiator ID which we think
will be the same as the Access ID in Hafner's proposal.

Multiple sessions can be used to separate different
classes of traffic (bulk vs. latency sensitive)
or even multiple processes

-------------
Deadlock:

Luciano pointed out that it is possible to run out of
buffers and deadlock with multiple TCP connections.

The source of the problem is 
        1) receive too many out-of-order commands
        2) receiving too much unsolicited (immediate)
           data

The solution to 1) is to either
   - limit the number of out-of-order commands that
     are read from each TCP pipe to 1 (requires NIC
     to know that command is out-of-order) and then
     stop reading from the connections (deskewing)
   - have a windowing mechanism on the command
     ordering queue in target
   - have a separate TCP pipe for emergency 
     recovery commands
   - Nuspeed aborts command with SCSI status TASK QUEUE FULL

The consensus seems to have resulted in windowing
being adopted.

The consensus solution to 2) was to allow the
target to drop immediate data and request it be
retransmited via ready-to-transmit (RTT).

--------------

Should task management commands be ordered with respect to tasks?

Those against feared that ordering task mangement commands
would prevent their timely delivery.

Those for feared that not ordering task management commands
would lead to surprising behaviors (like ABORT TASK SET 
overtaking and not aborting all previously issued tasks).

----------------

Can a single iSCSI TCP connection use multiple paths in the network
simultaneously?

Answer: Most networks keep a flow on one path to help ensure
minimal re-ordering, so no in that case. Of course, this being IP,
people could design a network that sprays packets of a flow across
multiple paths and it would still work...