SORT BY:

LIST ORDER
THREAD
AUTHOR
SUBJECT


SEARCH

IPS HOME


    [Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

    RE: iSCSI: No Framing



    Jim
    
    I'd like to add our input to the "live discussion" (hoping we can reach
    broader consensus and closure in the Interim):
    
    We've analyzed the various proposals for long time. As the fundamentals of
    iSCSI is transmitting  messages on top of a byte stream mechanism i.e. TCP,
    we share the views it needs to be addressed in a standard way. The preferred
    way to do it has been a generic Framing on top of TCP, but the recent IETF
    decisions has made it clear that this will not be available for some time,
    at least not on top of TCP.
    
    Therefore, we're back to square zero. iSCSI is better served with a Framing
    support that enjoys vendors support and that can also prevent a slow
    Initiator from placing high buffering requirements on a fast Target, running
    at 1Gb/s or 10Gb/s. So my conclusion has to be that IPS is better off with
    adoption of Markers that are not hard for HW or SW.
    
    FIM vs. COWS: FIM is preferred. COWS will drag the group to even more
    discussions on mode selection and impact on SW Initiators. We should leave
    room for negotiation of another Framing mechanism in the future, that may be
    the result of the TSVWG group efforts.
    
    SHOULD vs. MAY vs. MUST: we may have enough support in the WG for a MUST,
    but it seems to me that the bare minimum is to make sure FIM is supported
    without introducing interoperability risks. But if there isn't enough
    support for a MUST, at least we need to make sure the spec is very clear on
    Framing. Hence, the bare minimum is that Framing support is integrated into
    the Login negotiation and the all mechanics nailed down (e.g. FIM
    initialization, edge cases e.g. Marker inside the iSCSI PDU header etc.)
    
    thx
    
    Uri
    -----Original Message-----
    From: WENDT,JIM (HP-Roseville,ex1) [mailto:jim_wendt@hp.com]
    Sent: Tuesday, January 29, 2002 10:47 PM
    To: ips@ece.cmu.edu
    Subject: iSCSI: No Framing
    
    
    Perhaps we should discuss the possibility of not 
    specifying any framing mechanism (FIM or COWS) in the 
    first version of iSCSI.
    
    The arguments for not specifying framing include 
    (there may be others as well):
    1) The brute force approach of putting TCP receiver 
    reassembly memory on the iSCSI NIC will work for both 
    1Gbps and 10Gbps. Cost is incurred for memory chips, 
    ASIC pins, power, and board space. But, it is a 
    feasible approach.
    
    2) I/O bus latency (or bandwidth limitations) could 
    mandate inbound buffering (as a 'smoothing' buffer) 
    from the iSCSI NIC to the host system. If this buffer 
    memory is large enough to have to be off-chip, then 
    it will require some minimum number of memory chips 
    to provide the necessary bandwidth, and the 
    memory/pins/power/space costs will be incurred 
    anyway.
    
    3) Very large receive buffering on the NIC is only 
    required for high-bandwidth links traversing large 
    geographic distances (which may not be the 
    predominate case). These specialized implementations 
    can cost more (e.g. more memory/power/pins/etc) and 
    customers will likely be willing to pay accordingly.
    
    4) Many NICs will likely aspire to be combo 
    iSCSI+TOE+Ethernet NICs allowing the host system to 
    use a single Ethernet port for all of its network 
    communications. The TOE function on this combo NIC 
    will already require TCP receive buffering and off-
    chip buffer memory to support the existing sockets 
    interface receive model.  More importantly, to allow 
    senders to assume ownership of the buffer upon return 
    from a socket send call, the TOE NIC would need to 
    copy application send buffers into NIC memory as 
    well.
    
    5) The framing/marker mechanism will be an iSCSI-only 
    solution. It is quite possible that the problem will 
    be solved via a different, and hopefully more 
    general, mechanism for other ULPs.
    
    6) Both FIM and COWS are poor solutions for software 
    implementation. COWS requires, at a minimum, the 
    sender to read every word in the buffer. FIM either 
    imposes additional sender gather processing (iovecs) 
    or requires a data buffer copy (on systems that don't 
    support HW gather on sends).
    
    7) Unless all senders thoroughly test framing/markers 
    now (i.e. unless the framing method is a MUST 
    implement), there is potential for future 
    interoperability problems as framing/marker receivers 
    are deployed in the future.
    
    8) Neither FIM nor COWS is an elegant solution. Are 
    we comfortable with the legacy we are creating under 
    the umbrella of state-of-the-art IP networked 
    storage?
    
    9) Is it essential to build in forward compatibility 
    now, or will there be a different solution in the 
    10Gbps or 40Gbps timeframe - perhaps involving iSCSI-
    2?  Will it be reasonable to update or bridge initial 
    1Gbps deployments?
    
    
    So, it would be good to hear from several iSCSI 
    NIC/chip implementors who:
    - have or plan to implement FIM or COWS (or some 
    other framing mechanism) and take advantage of it to 
    minimize demands on on-NIC buffer memory 
    bandwidth/quantity.
    - believe that all-buffers-on-chip solutions are 
    feasible and valid (wrt the points above, including 
    #2)
    - can quantify the memory/pin/power/space cost 
    savings for all-buffers-on-chip solutions
    
    Jim
    
    


Home

Last updated: Tue Feb 05 20:18:08 2002
8657 messages in chronological order