RE: Avoiding deadlock in iSCSI

[julian_satran@il.ibm.com]

Correct. And lack of queue space should lead to a state in which all the
commands are rejected until the state is explicitly restored.

Consider also that in the asymmetric case with TCP in charge of the window
and only small amounts of immediate data that is very unlikely to happen.

Julo

"John Hufferd/San Jose/IBM" <hufferd@us.ibm.com> on 12/09/2000 09:50:05

Please respond to "John Hufferd/San Jose/IBM" <hufferd@us.ibm.com>

To:   ips@ece.cmu.edu
cc:    (bcc: Julian Satran/Haifa/IBM)
Subject:  RE: Avoiding deadlock in iSCSI




Costa,
I think I understand your points, but please also consider the Asymmetric
approach.  Please consider the Asymmetric approach and also as I said in my
last note,  suppose that we "make a rule, that no unsolicited data can be
sent, before its command is sent AND that unsolicited data must be sent in
the same order as the commands that reference it.  Along with that rule,
and the statements below from David -- will, in your opinion, the
Asymmetric approach work even with unsolicited data?

.
.
.
John L. Hufferd

---------------------- Forwarded by John Hufferd/San Jose/IBM on 09/11/2000
11:19 PM ---------------------------

David Robinson <David.Robinson@EBay.Sun.COM>@ece.cmu.edu on 09/11/2000
06:52:22 PM

Please respond to David Robinson <David.Robinson@EBay.Sun.COM>

Sent by:  owner-ips@ece.cmu.edu


To:   ips@ece.cmu.edu
cc:
Subject:  RE: Avoiding deadlock in iSCSI



Thanks for your clarifying comments.

> > In general
> > I consider that to be a bug and the receiver should just drop the
> > data on the floor.
>
> Not as I understand solicit.

This I don't understand, it you get data that no command has been
sent yet, as the reciever you can either wait and hope the
command arrives consuming buffer space but eventually dropping
it after some timeout, or drop it on the spot. In either case this
seems like a bad design we should try to avoid.

> > My first assumption is that the sender would not send commands
> > C1 and C2 and data D2 and D1 on the same connection. Doing that
> > creates nasty ordering problems we want to avoid.
>
> Order on the wire can not be controlled. Only the ULP can avoid such.

Yes it can, this is exactly the advantage of using a reliable stream
protocol like TCP, the session layer never sees out of order packets.
With multiple data connections and the appropriate ordering constraints
we have no deadlock or buffer management issues.

> Resources are held until associated data is received to complete
operations.
> If the resource limit is not the data buffer nor freed by content already
> within the data buffer, this will result in discarding commands.

But with a reliable stream no commands need to be discarded, the
transport flow controls so the commands are held at the sender.

> > With multiple data connections, some may flow
> > control but the active command will be able to make progress on
> > one connection. This may not be the most efficient mechanism but
> > it is "safe".
>
> One connection per LUN or one connection per command, safe but expensive?

Define "expensive", not in terms of performance as one TCP
connection can saturate the link layer or not in terms of memory
as the mux/demux state has to be held either in the transport or
the session layer.  I am not advocating a connection per command
as that is just a bad datagram protocol, but either a connection per
LUN or per target should work just fine.

> As a means for freeing resources, data is to be discarded within the
iSCSI
> architecture.  As such, even unsolicited data may be requested by the
> target.

I don't understand this statement. Short of target errors or connection
errors, why does data need to be discarded? The sender should never
send data without a command, and on a given connection the data MUST
always be sent after the corresponding command and data from two
commands must always be sent in order if on the same connection.

     -David