RE: iSCSI: Out of order commands

["John Hufferd" <hufferd@us.ibm.com>]

I guess I disagree with this design approach.  The buffers on the iSCSI
HBAs are for Eddy buffers.  Once you have the iSCSI Header should be able
to place the data directly into a Shared buffer.  If you have a session
that spans HBAs, that should work just fine.  But the buffers match the
Session Window, and the Buffers need to be shared across the HBAs, not in
the HBAs.  Again, the HBAs should just have the Eddy Buffers.

.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
Home Office (408) 997-6136, Cell: (408) 499-9702
Internet address: hufferd@us.ibm.com


"Somesh Gupta" <somesh_gupta@silverbacksystems.com> on 11/08/2001 04:31:29
PM

Please respond to <somesh_gupta@silverbacksystems.com>

To:   John Hufferd/San Jose/IBM@IBMUS
cc:   "Robert D. Russell" <rdr@mars.iol.unh.edu>, Julian
      Satran/Haifa/IBM@IBMIL, <ips@ece.cmu.edu>
Subject:  RE: iSCSI: Out of order commands



John,

You are correct if all the connections are on
a single adapter. In this case the buffer pool
can be shared among all the connections on the
adapter.

However this is not true if the connections are
on different adapters (the point being discussed).
Think of there being a big buffer pool in the board
into which multiple adapters/chips are "plugged" in.
However, at any given point in time, these buffers
are allocated (handed over through command queues or
whatever) to each of the individual adapters. And this
is where the problem arises.

The assumption that adapters can share a typical host
command buffer pool on their own without host intervention
is incorrect.

Somesh

> -----Original Message-----
> From: John Hufferd [mailto:hufferd@us.ibm.com]
> Sent: Thursday, November 08, 2001 4:20 PM
> To: somesh_gupta@silverbacksystems.com
> Cc: Robert D. Russell; Julian Satran; ips@ece.cmu.edu
> Subject: RE: iSCSI: Out of order commands
>
>
>
> Most folks I know have a buffer that applies to all the
> connections, not to
> just one buffer per connection.  I fail to see the problem for most
> implimentations.  The window should apply to the session.
>
> .
> .
> .
> John L. Hufferd
> Senior Technical Staff Member (STSM)
> IBM/SSG San Jose Ca
> Main Office (408) 256-0403, Tie: 276-0403,  eFax: (408) 904-4688
> Home Office (408) 997-6136, Cell: (408) 499-9702
> Internet address: hufferd@us.ibm.com
>
>
> "Somesh Gupta" <somesh_gupta@silverbacksystems.com>@ece.cmu.edu on
> 11/08/2001 02:22:30 PM
>
> Please respond to <somesh_gupta@silverbacksystems.com>
>
> Sent by:  owner-ips@ece.cmu.edu
>
>
> To:   "Robert D. Russell" <rdr@mars.iol.unh.edu>
> cc:   Julian Satran/Haifa/IBM@IBMIL, <ips@ece.cmu.edu>
> Subject:  RE: iSCSI: Out of order commands
>
>
>
> Comments below
>
> > -----Original Message-----
> > From: Robert D. Russell [mailto:rdr@mars.iol.unh.edu]
> > Sent: Thursday, November 08, 2001 11:46 AM
> > To: Somesh Gupta
> > Cc: Julian Satran; ips@ece.cmu.edu
> > Subject: RE: iSCSI: Out of order commands
> >
> >
> >
> > Somesh:
> >
> > > > It seems to me that if a target can only handle x new commands,
> > > > then its queueing capacity is x, and in the SCSI Response PDU it
> > > > should set MaxCmdSN = (ExpCmdSn + x - 1), in accordance with the
> > > > formula in section 2.2.2.1.  This in turn controls the number of
> > > > commands the initiator can send, and thus prevents the incoming
> > > > commands from overfilling the target's queue.
> > >
> > >   This is actually a weakness of the multiple connections over
> > >   multiple adapters model (it is not related to ordering).
> > >   The target advertises a command window on a session wide basis.
> > >   At the same time, it has to provide the resource to the adapter
> > >   to be able to DMA those commands to. Since there is no restriction
> > >   on which connection the commands may be received, either the
> > >   target has to allocate the max resources needed to each adapter
> > >   (thus committing n times the resources actually required), or
> > >   has to "lie" (it would not be a complete lie) which could
> > >   result in running out of resources. One way to fix that would be
> > >   to have the credit on a per connection basis.
> >
> > If I understand you correctly, you are saying that deadlock can
> > occur even if we enforce ordered delivery by initiators and even
> > if the advertisements are correct and even if both parties obey
> > the advertisements.
>
>   I would let Julian enlighten us on that, or whether it just
>   leads to wholesale command drops and retries, or TASK SET FULL
>   is returned (is TASK SET FULL a valid response in this case
>   for a LINKED task when it is not the first command?)
>
> >
> > In other words, doesn't this mean that the standard can lead to a
> > trap in implementing targets and that, in fact, the advertisements
> > really should be on a per connection basis?
> > However, what would be the consequences for initiators if
> > the advertisements were on a per-connection basis?
>
>   There is really no issue with adveritisement on a per
>   connection basis. It leads to a protocol change. However, it
>   also allows a much more performant flow of commands.
>
> >
> >
> > Thanks,
> >
> > Bob Russell
> > InterOperability Lab
> > University of New Hampshire
> > rdr@iol.unh.edu
> > 603-862-3774
> >
> >
> >
>
>
>
>