RE: iSCSI: Out of order commands

["Somesh Gupta" <somesh_gupta@silverbacksystems.com>]

Bob,

> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Robert D. Russell
> Sent: Thursday, November 08, 2001 7:51 AM
> To: Julian Satran
> Cc: ips@ece.cmu.edu
> Subject: RE: iSCSI: Out of order commands
>
>
> Julian:
>
> > However allowing initiators to ship them out of order creates a
> > potential deadlock that does not appear otherwise.
> >
> > If a target is missing a command in a queue (and there are no
> errors) then
> > this command is bound to be first on some connection under the
> current set
> > of rules.
> >
> > If we allow OOO shipping then the missing command can be somewhere
> > "inside" the window on some connection and if the target has
> just filled
> > his queue and has room in the staging buffer only for the command it is
> > waiting for and that command happens to be the first to pass to
> SCSI   you
> > have a deadlock.
>
>
> I must be understanding something.
>
> Your example is certainly correct if the target has no control
> over the number of commands sent to it by the initiator.
> But the target does control this number through the MaxCmdSN field.
> For the scenario you are describing to occur, wouldn't it be
> necessary for the target to advertize a MaxCmdSN value bigger than
> it actually has resources to handle?
>
> It seems to me that if a target can only handle x new commands,
> then its queueing capacity is x, and in the SCSI Response PDU it
> should set MaxCmdSN = (ExpCmdSn + x - 1), in accordance with the
> formula in section 2.2.2.1.  This in turn controls the number of
> commands the initiator can send, and thus prevents the incoming
> commands from overfilling the target's queue.

  This is actually a weakness of the multiple connections over
  multiple adapters model (it is not related to ordering).
  The target advertises a command window on a session wide basis.
  At the same time, it has to provide the resource to the adapter
  to be able to DMA those commands to. Since there is no restriction
  on which connection the commands may be received, either the
  target has to allocate the max resources needed to each adapter
  (thus committing n times the resources actually required), or
  has to "lie" (it would not be a complete lie) which could
  result in running out of resources. One way to fix that would be
  to have the credit on a per connection basis.

>
> So isn't the deadlock caused by a broken target?
> Isn't the target advertizing a queueing capacity greater than it
> actually has and isn't that the cause of the deadlock?
>
> An alternative explanation of the deadlock might be that the
> target is advertizing the correct MaxCmdSN, but the initiator is
> sending commands beyond what it is allowed to send.
>
> However, in this case the target should silently ignore any
> non-immediate command outside the allowed range.
> For the deadlock to occur, wouldn't the target have to be queuing
> commands outside the allowed range instead of ignoring them?
>
> So in this case too, the target is broken and that is what causes
> the deadlock.
>
> Where am I going wrong in this reasoning?
>
> Thanks,
> Bob Russell
> InterOperability Lab
> University of New Hampshire
> rdr@iol.unh.edu
> 603-862-3774
>
>