SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    iSCSI : Command Ordering Proposal.


    • To: IPS Reflector <ips@ece.cmu.edu>
    • Subject: iSCSI : Command Ordering Proposal.
    • From: Santosh Rao <santoshr@cup.hp.com>
    • Date: Thu, 25 Jan 2001 18:15:15 -0800
    • Content-Type: multipart/mixed;boundary="------------4BAD780921B262B30E6FF7C2"
    • Organization: Hewlett Packard, Cupertino.
    • Sender: owner-ips@ece.cmu.edu

    Julian & All,
    
    Proposal :
    =======
    CmdSN should be used to enforce ordering only when the Ordered Task Tag
    attribute is set in the SCSI Command PDU.
    
    Reference :
    =========
    Current iSCSI model for enforcing command ordering, based on striping
    commands across multiple TCP connections in an iSCSI Session and using a
    
    CmdSN to order the commands received out of order. Currently, CmdSN is
    used to enforce ordering for all commands.
    
    Objective of multi-connection sessions :
    ==============================
    Multiple TCP connections per session were defined in order to
    allow for :
    a) bandwidth aggregation by the creation of a fat pipe between the
    target and initiator.
    b) to optimize against single connection failures.
    c) to allow for iSCSI layer failover mechanisms within a session.
    
    
    Problems with the current ordering model :
    ================================
    1) When one TCP connection is faulty in an iSCSI session and is causing
    command transmission failures, all the commands being sent on other
    connections in the session will stall [due to the CmdSN ordering]
    at the receiving iSCSI layer of  the target, until TCP
    timeouts + re-transmissions for the commands on the faulty
    connection are exhausted and a "link ping" of the faulty
    connection fails, thereby, detecting connection failure and initiating
    connection failure recovery.
    
    While operating under such conditions, the commands sent on
    other TCP connections within the session will be stalled until the
    initiator commences connection failure recovery and re-sends the
    commands with their original CmdRN on a new connection.
    
    This can result in a large number of stalled I/Os that can remain
    stalled for the period it takes to detect a faulty TCP connection and
    complete connection failure recovery.
    
    2) This strict command ordering introduces a lot of complexity into
    error recovery conditions like connection failures, QUEUE FULL, BUSY &
    CHECK CONDITION handling due to the attempts iSCSI makes to enforce
    strict ordering of command delivery.
    
    3) This model depends on the retry of ALL the failed commands on a new
    connection in order to bridge the missing CmdSNs at the target. If a
    subset of the failed commands were not to be retried [due to a retry
    policy that forbids this, such as non-idempotent media, or a
    ULP timeout],
    then, the CmdSNs at the target waiting for the out-of-order commands to
    arrive, need bridging which results in additional complexity.
    
    Requirements for Ordering :
    =====================
    1) SCSI ordering is only required on a per-LUN basis.
    
    2) Strict SCSI ordering is only required when requested through the
    Ordered Task Tag attribute. For all the other types of tasks, no command
    
    ordering is required. Strict Command ordering is not enforced in other
    SCSI transport protocols. Command ordering in other SCSI transport
    protocols is violated when a QUEUE FULL, BUSY or CHECK
    CONDITION SCSI Status is returned. Command ordering can also
    be violated when the SCSI transport encounters transport failures
    or resource allocation failures on intermittent commands.
    
    3) SAM Section 4.6.2 assumes the service delivery subsystem to provide
    in-order delivery for convinience. "This assumption is only made to
    simplify
    description of behaviour and does NOT constitute a requirement."
    
    Solution :
    ========
    iSCSI should only enforce strict ordering when requested to do so by the
    
    SCSI ULP through the Ordered Task Tag attribute.
    
    Benefits :
    ========
    1) Optimizes iSCSI performance by avoiding stalls on other TCP
    connections within a session when one connection is faulty.
    
    2) Simplifies error recovery on connection failure, digest error,
    CHECK CONDITION, QUEUE FULL and BUSY scsi status as
    well as initiator resource allocation errors.
    
     -------------------------------------------------------
    
    Regards,
    Santosh
    
    begin:vcard 
    n:Rao;Santosh 
    tel;work:408-447-3751
    x-mozilla-html:FALSE
    org:Hewlett Packard, Cupertino.;SISL
    adr:;;19420, Homestead Road, M\S 43LN,	;Cupertino.;CA.;95014.;USA.
    version:2.1
    email;internet:santoshr@cup.hp.com
    title:Software Design Engineer
    x-mozilla-cpt:;21088
    fn:Santosh Rao
    end:vcard
    


Home

Last updated: Tue Sep 04 01:05:41 2001
6315 messages in chronological order