RE: Avoiding deadlock in iSCSI

[David Robinson <David.Robinson@EBay.Sun.COM>]

Thanks for your clarifying comments.

> > In general
> > I consider that to be a bug and the receiver should just drop the
> > data on the floor.
> 
> Not as I understand solicit.

This I don't understand, it you get data that no command has been
sent yet, as the reciever you can either wait and hope the
command arrives consuming buffer space but eventually dropping
it after some timeout, or drop it on the spot. In either case this
seems like a bad design we should try to avoid.

> > My first assumption is that the sender would not send commands
> > C1 and C2 and data D2 and D1 on the same connection. Doing that
> > creates nasty ordering problems we want to avoid.
> 
> Order on the wire can not be controlled. Only the ULP can avoid such.

Yes it can, this is exactly the advantage of using a reliable stream
protocol like TCP, the session layer never sees out of order packets.
With multiple data connections and the appropriate ordering constraints
we have no deadlock or buffer management issues.

> Resources are held until associated data is received to complete operations.
> If the resource limit is not the data buffer nor freed by content already
> within the data buffer, this will result in discarding commands.

But with a reliable stream no commands need to be discarded, the
transport flow controls so the commands are held at the sender.

> > With multiple data connections, some may flow
> > control but the active command will be able to make progress on
> > one connection. This may not be the most efficient mechanism but
> > it is "safe".
> 
> One connection per LUN or one connection per command, safe but expensive?

Define "expensive", not in terms of performance as one TCP
connection can saturate the link layer or not in terms of memory
as the mux/demux state has to be held either in the transport or
the session layer.  I am not advocating a connection per command
as that is just a bad datagram protocol, but either a connection per
LUN or per target should work just fine.

> As a means for freeing resources, data is to be discarded within the iSCSI
> architecture.  As such, even unsolicited data may be requested by the
> target.

I don't understand this statement. Short of target errors or connection
errors, why does data need to be discarded? The sender should never
send data without a command, and on a given connection the data MUST
always be sent after the corresponding command and data from two
commands must always be sent in order if on the same connection.

	-David