SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: Flow Control



    
    
    Pierre,
    
    a) This group never agreed on such a thing. There is a rough agreement
    that the amount should be limited. And again keep in mind that we talking
    about two types of unsolicited data
    - immediate an no-immediate. The only thing we sort of agreed is bad is to
    allow immediate data when all data are required to be solicited by the
    appropriate configuration
    bit.
    
    b) flow control on data is there with both SCSI and TCP taking care of it.
    
    c) it was repeatedly stated that you can't avoid task-set-full as this is
    an individual LU issue. If you are suggesting flow-control per LU and
    per initiator I guess everybody in this space will tell you that this is
    expensive and barely useful.
    
    d) I agree that would be a desiderata but perhaps you can hand as the key
    piece missing - how to do it!
    
    Julo
    
    Pierre Labat <pierre_labat@hp.com> on 06/10/2000 19:00:08
    
    Please respond to Pierre Labat <pierre_labat@hp.com>
    
    To:   IPS@ece.cmu.edu
    cc:    (bcc: Julian Satran/Haifa/IBM)
    Subject:  RE: iSCSI: Flow Control
    
    
    
    
    
    
    
    Hi,
    
    From the mails on this thread it seems that most
    people agree on the following ?
    
    A) Unsolicited data will be allowed by iSCSI.
    Well, i think that the protocol must be able to handle
    large quantities of unsolicited data to be performant
    for the writes on large latency networks.
    
    B) We need of course a flow control on the data
    because even if we have a flow control on the commands,
    just a few commands with huge unsolicited data could
    overwhelm the target buffers.
    
    C) We need a flow control on the commands to avoid
    the TASK SET FULL condition. If the target hit this condition
    it has to drop command (return task set full) and
    drops data associated. Avoiding the task set full condition
    avoid too the "dead locks".
    
    D) As noted by several, one TCP connection doesn't "block"
    the commands behind the data. The iSCSI adapters (on the
    initiator side) must be build in a way to send first the
    commands/task management requests. As soon as a command/task mgt
    is posted to the adapter, the next iSCSI PDU sent by
    the adapter will be the one encapsulating the command/task mgt
    request. This has to be true for any command (read/write...)
    and task mgt request.
    
    
    About managing these flow controls:
    ==================================
    The TCP window can handle efficiently the flow control for the data.
    Now we need to flow control the commands. The solutions
    can be:
    
    1) Use the window described in the draft
    
    Advantage:
    ---------
     o Flow control the commands
    
    Disavantages:
    -------------
     o Some said that as the flow control is per session,
       the target will flow control the whole session
       (several LUs) when the task set of one LU will
       become [almost] full. Hence one LU blocks the other
       ones in the session when they could continue to work.
    
       This problem can be workaround by the
       target using a right command blocks allocation policy.
       (A command block is a buffer containing the command
        pending (in the task set) in the target).
    
       Instead of having a policy where the command  blocks
       are allocated per LU, the target could use a policy
       where the command blocks are allocated per session.
       The target maintains a pool of blocks per session not per
       LU. In this case one LU cannot block another one.
       That means that inside a session the balance of the commands
       in the various task sets is driven by the initiator not
       by the target.
       It doesn't seems to be wrong doing that, hence this disavantage
       is NOT a disavantage.
    
     o If the initiator send a mix of fast and slow commands the slow
       commands can close the command window even if the target have
       slots free for new commands.
    
    2) Replace the window in the iSCSI draft with a command credit.
       The target advertises (with each response) a credit to the initiator.
       The command reference number and status reference numbers
       can be kept for ordering/recovery purpose. Now, the MaxCMdRN
       becomes a credit unrelated to the CmdRn, that's the only change compared
       to the draft. It specifies the maximum number of commands in flight over
       the session.
    
    Advantage
    ---------
     o flow control commands
     o get rid of the blocking of fast commands by slow commands.
    
    Disavantage
    -----------
     o None
    
    
    
    3) Use a (iSCSI) credit per LUN (advertised with the command responses)
    
    Advantage
    ---------
     o flow control the commands (per LUN)
    
    Disavantage
    -----------
     o the iSCSI layer on the initiator needs to maintain a state and a flow
    control
       per LUN
    
    
    
    4) T10 specifies a command credit advertising mechanism (credit returned
       with the status + some asynchronous advertisements (see Mike mail/IBA))
    
    Advantage
    ---------
     o flow control the commands (per LUN)
     o the state of the flow control would be maintained at the SCSI layer,
       it's ok because the SCSI layer already mantains some states/LUN
     o will benefit to FC too
    
    Disavantage
    -----------
     o are we able to persuade T10 to specify that quickly?
     o need to change the SCSI layer
    
    
    Conclusion
    ==========
    It seems to me that with 2) and even with only one TCP connection,
    we have a solution very performant in term of throughput, simple,
    and with no dead locks.
    Do you agree?
    
    
    Regards,
    
    
    Pierre
    
    
    
    


Home

Last updated: Tue Sep 04 01:06:47 2001
6315 messages in chronological order