Re: iSCSI: Towards a more effective PDU format

[Matt Wakeley <matt_wakeley@agilent.com>]

Barry,

The objective of the "fixed header" with all the "optional" fields in it was
to improve the performance of software implementations of iSCSI.  With the
fixed header, software could to a single "read()" and get the whole header -
everything (reads can incur context switch overhead into kernel space).  Then
it could to another "read()" to get the data (if any).

I also don't like your suggestion of separate digests for optional headers. 
There is precedent in the IP header that options are covered under the same
"check" as the rest of the header.

As for the "whats next" headers, I understand that Julian was just trying to
reduce the overhead of iSCSI on the link.  I think a complex item is ok if it
is clearly and simply documented (which it is currently not).

-Matt Wakeley
Agilent Technologies

Barry Reinhold - Lamprey POP3 wrote:
> 
> All,
> 
> I know there are well defined feelings about how a PDU should be structured,
> and not everyone is going to be in the same camp on this issue. However, I
> am concerned that the current PDU format with the WN field has not really
> achieved the goals that I, for one, would like to see it achieve. So, let me
> list a few objectives for the PDU format and then suggest one possible
> solution.
> 
> Objectives for the iSCSI PDU (that could be improved upon).
> 
> 1. Simplicity - The PDU layout should be as straight forward as possible.
> Interoperability starts with simplicity. When the logical function is
> simple, documenting proper usage is a whole lot easier. When documentation
> is clear in a standard, the time to interoperability and market acceptance
> is reduced.
> 
> 2. Efficient operation in the common execution path. I think "rough
> consensus" can probably be achieved in terms of what parts of an iSCSI
> header are going to be needed in the normal flow and what parts are
> exception cases. If we have consensus on this we should be able to agree on
> where to make things easy and where to add complexity.
> 
> 3. Data integrity - If a header digest is going to be provided, the
> information it covers should be checked against the digest before being
> used.
> 
> I think we can improve the current PDU format, in terms of these objectives,
> by doing the following:
> 
> 1. Providing a fixed size header that contains the most commonly used
> fields.
> 2. Placing less commonly used fields in an optional portion of the header.
> 3. Providing a header digest over the fixed portion of the header and a
> separate digest for the optional portion of the header.
> 
> Basic approach:
> 
> 1. The fixed "BHS" portion of the header is in all iSCSI PDU headers. It is
> always the same size and is followed by a header digest that covers the
> "BHS", if header digests are being used. The fact that header digests are
> being used within a TCP byte stream is not encoded into the frame, but is
> obtained from state associated with the session. This allows the location of
> the header digest to be determined without using any information in the
> header. Only when the integrity of the header has been validated is the
> information in it used.
> 
> 2. The header contains information on the size of the complete PDU, and a
> length field for any additional headers. The integrity of this information
> has been validated as it is part of the "BHS". The location of the
> additional headers, if they exist (in most cases they will not) is after the
> header digest. Thus additional headers can be located using validated
> information.
> 
> 3. Each additional header is encoded using a standard TLV (type, length,
> value) format. In TLV format the fixed size type indicator tells how to
> interpret the data. The fixed size length field says how long the data is,
> and the variable sized value field contains the data. If an application
> doesn't understand the type, it can maintain sync. in the byte stream by
> skipping to the length field and then skipping the value.
> 
> 4. The complete set of TLV values are covered by a distinct digest. This
> digest is considered a part of the header digest in terms of iSCSI option
> negotiation but is physically distinct. Although this requires another
> digest calculation its usage is expected to be fairly low and it is most
> likely calculated over a small number of bytes.
> 
> 5. The data and data digest follow the TLV digest.
> 
> In terms of layout, what you might have is the following:
> 
> ================= Start of iSCSI PDU =============================
> 
>         This portion of the PDU contains the information in
>         the "BHS" plus:
> 
>         1. The length field (4 bytes)
>         2. The length of the TLV information
> 
> --------------- End of the fixed size header ---------------------
> --------------- Start of the header digest (optional) ------------
> 
> --------------- End of the header digest -------------------------
> --------------- Start of TLV information (optional) --------------
> 
>     (Additional header fields go here in a variable length space)
> 
> --------------- End of TLV information ---------------------------
> ------ Start of "TLV header" digest (if header digest present)----
> 
> --------------- End of "TLV header" digest -----------------------
> --------------- Start of data ------------------------------------
> 
> --------------- End of data --------------------------------------
> --------------- Start of data digest (optional) ------------------
> 
> =============== End of data digest and PDU =======================
> 
> [David B: This looks better in a .pdf....are we allowed to use pdfs on this
> reflector?]
> 
> I believe this to be a simple yet efficient structure for common usage.
> The optional headers require more involved processing, but allow incremental
> advancement of the TCP byte stream and avoids the loss of synchronization
> that might take place in other formats.
> The header can be verified, before being used.
> 
> [Note 1: This PDU format does not attempt to address the problem of how to
> recover when there is a digest error. I believe this to be a distinct
> problem, though the PDU format could provide some additional level of
> robustness via redundancy of information at known locations.]
> 
> [Note 2: Julian, If more detailed information on the PDU format is needed I
> would be glad to present it. However I think that most of the significant
> issues are addressed by this outline. The other items don't appear, to me,
> to be as significant.]
> 
> Barry Reinhold
> Barry.Reinhold@trebia.com
> 603-659-0885