Re: iSCSI: current UNH Plugfest

["Julian Satran" <Julian_Satran@il.ibm.com>]

Dave,


It all depends on what assumptions you make about data buffering for 
unsolicited data.
If you assume (conservatively) that you will have enough buffers for all 
possible commands in flight and their unsolicited data
then ordering is not an issue.
If you assume - less conservative - that you have do not have room for all 
and account for the "flight time" to make up for some of the missing 
buffers and perhaps keep some of the data in the "TCP buffers" then 
allowing commands out of order might result in deadlock if the source and 
sink have different ordering.

As shipping commands out of order will hardly get you a real performance 
boost and keeping things in order might have
beneficial effects on the memory sizes required at target I thought that 
we should require ordering.

Regards,
Julo





Dave Sheehy <dbs@acropora.rose.agilent.com>
Sent by: owner-ips@ece.cmu.edu
02-11-01 23:18
Please respond to Dave Sheehy

 
        To:     ips@ece.cmu.edu (IETF IP SAN Reflector)
        cc: 
        Subject:        Re: iSCSI: current UNH Plugfest

 


> 3. Can commands be sent out of order on the same connection?
> 
>    The behavior of targets is clearly specified in Section 2.2.2.3 on
>    page 25 of draft 8, which says:
>      "Except for the commands marked for immediate delivery the iSCSI
>      target layer MUST eliver the commands for execution in the order
>      specified by CmdSN."
> 
>    Section 2.2.2.3 on page 26 of draft 8 also says:
>      "- CmdSN - the current command Sequence Number advanced by 1 on
>      each command shipped except for commands marked for immediate
>      delivery."
>    but the meaning of the term "shipped" is vague, and does not 
> necessarily
>    require that the PDUs arrive on the other end of a TCP connection
>    in the same order that the CmdSN values were assigned to these PDUs.
> 
>    Some initiators have been designed to send commands out of CmdSN
>    order on one connection.  Consider the situation where there is only
>    one connection and a high-level dispatcher creates a PDU for a SCSI
>    command that involves writing immediate data to the target.  This PDU
>    is enqueued to a lower-level layer which has to setup, start, and
>    wait-for a DMA operation to move the immediate data into an onboard
>    buffer before the PDU can be put onto the wire.  While this is
>    happening, the dispatcher creates another unrelated PDU for a SCSI
>    read command (for example), and when this PDU is passed to the
>    lower-level layer it can be sent immediately, ahead of the previous
>    write PDU and therefore out of order on this connection.
> 
>    The standard clearly allows this to happen if the two PDUs were sent
>    on different connections, and seems to imply that this can also 
happen
>    when the two PDUs are sent on the same connection.
> 
>    The suggestion is to put in the standard an explicit statement that
>    this is allowed or not allowed, as appropriate.
> 
>    If this is allowed, such a statement would avoid the erroneous
>    assumption being made by some target implementers that within a 
single
>    connection, commands will arrive in order.
> 
>    If this is not allowed, such a statement would avoid the erroneous
>    assumption being made by some initiator implementers that within a
>    single connection, commands can be put on the wire out of order.
> 
> +++ 
> 
> will add an explicit statement saying that this behaviour is forbidden.
> 2.2.2.1 will contain:
> 
> On any given connection, the iSCSI initiator MUST send the commands in 
the 
> order specified by CmdSN.
> 
> +++

Why do you feel this behavior should be forbidden? Targets already have to
order commands across the session. I don't see why it's a problem to 
extend
that to the connection as well. I, for one, believe we should take a 
liberal
stance on this.

Dave Sheehy