Re: iSCSI: Towards a more effective PDU format

["Robert D. Russell" <rdr@mars.iol.unh.edu>]

Julo:

Even if we give up on the idea of having a second header digest, I still
believe that some changes should to be made to the PDU Format proposed in
version 5 to make it simpler, more efficient to process, and more robust.

I would therefore propose eliminating the use of the current WN Next-Qualifier
scheme and replacing it as follows (this is just a slight modification of my
earlier proposal in order to eliminate the second header digest).

1. Every PDU must start with a 48-byte Basic Header (BHS) that can be read
   in a single "read" operation.  The format of this is identical to that
   proposed in section 2.2.4 of version 5, but with the addition of 2
   fields (4-bytes) as described next.

2. Every BHS contains 2 fields at fixed locations:

   a) AHS_length, containing the total length of all Additional Header
      Segments (AHS) that follow.  This field is 0 if there are no AHSs.

   b) DATA_length, containing the total length of all data that follows
      the AHSs.  This field is 0 if there is no data.

3. If the AHS_length field in the BHS is non-zero, all the AHSs immediately
   follow the BHS in the PDU.  The AHS_length value gives the total number of
   bytes in all the AHSs, allowing a single "read" operation to read them all.
   (The total header consists of 48+AHS_length bytes.)

4. If header digests are in use, one header digest covering the BHS and
   all the AHSs (if any) immediately follows the last AHS.  This digest
   does NOT require a separate read, since the first read (of the BHS)
   should be for 48+(length-of-header-digest) bytes.  If the AHS_length
   field is zero, there are no more reads.  If the AHS_length field is
   non-zero, exactly that many more bytes need to be read in a second read.
   These additional bytes are appended to those obtained in the first read.
   The header digest will always be the last (length-of-header-digest) bytes.

4. If the DATA-length field in the BHS is non-zero, all the data
   immediately follows the header digest.  If data digests are in use,
   a data digest immediately follows the data.

5. To save space within the BHS, the AHS-length field can be restricted to a
   single byte, and can be in units of 4-byte words.  This allows up to 1020
   bytes of AHSs to follow a BHS, which seems to be more than enough for the
   uses foreseen (so far, only extended CDB and bidi-read info).  Currently,
   byte 2 is unused (reserved) in all PDU types except Login and Login
   response, where byte 6 is unused.  Therefore, the AHS-length field can
   be added without increasing the BHS size of 48 bytes.

6. Each AHS should start with a word containing a TYPE field and a
   LENGTH field.  The TYPE field should be enumerated rather than
   bit-field encoded, for easier decoding and future expansion.
   The LENGTH field is the number of bytes of additional information
   in this AHS that follows this word.

7. The DATA_length field should be a 4-byte field added to the 44-byte
   BHS of the current section 2.2.4 to give a new BHS of 48-bytes.
   However, since the version 5 44-byte header is always preceded by
   a 4-byte WN Next-Qualifier that would no longer be needed, there is
   no net change in the effective BHS size of 48-bytes.


Advantages of this proposal over the current WN Next-Qualifer of version 5.

1. Because the receiver gets the AHS-length field from the BHS, it can obtain
   the entire set of ALL AHSs that follow the BHS in a single "read" operation
   (the current WN Next-Qualifier scheme requires a separate "read" for EACH
   additional header segment after the BHS).

2. By limiting the total size of all AHSs to 1020 bytes, a receiver can
   preallocate a fixed-length buffer of (48 + 1020 + size-of-header_digest)
   bytes for header processing (the current WN Next-Qualifier scheme has
   no limit on either total header size nor individual AHS sizes).

3. Since each AHS begins with a word containing the TYPE and LENGTH in
   fixed positions, an unknown TYPE (i.e., a type introduced in the
   future that is received by a legacy receiver) does NOT result in
   loss of synchronization during header processing -- the receiver
   knows the length of this AHS in any case, and can just skip over it
   (the current WN Next-Qualifier type determines how the length field
   is to be interpreted -- there is no "general rule", so unknown types
   mean loss of synchronization).

4. The format of the AHS is the familiar Type/Length/Value (T/L/V) structure
   (the current WN Next-Qualifier is T+1/L+1/V, but this does not seem to
   provide any benefit and only adds confusion and complexity -- see e-mails
   from David Black and Barry Reinhold).

5. The AHS TYPE is enumerated (the current WN Next-Qualifier is bit encoded,
   again without seeming to provide any benefit -- see e-mail from David Black).

6. Because the BHS always contains a 4-byte DATA_length field, the maximum data
   segment size is 4 Gigabytes (the current WN Next-Qualifier scheme, with
   the long data header removed, limits the data segment size to 16 Megabytes).

7. The "Header Digest Present" and "Data Digest Present" bits have been
   completely eliminated (the purpose of these in the current WN Next-Qualifier
   scheme was never explained, but they appear to be useless -- the presence
   or absence of digests has to be negotiated, and once negotiated, all
   PDUs must obey the negotiated decision).


I see no disadvantages of this proposal relative to the current WN scheme.


Since there is only one header digest, both schemes suffer the disadvantage
of using unreliable data to find the header digest, which can lead to
unnecessary blocking on "reads".


Proposed iSCSI PDU Format


     +------------------------+
     |     required BHS       | > fixed length of 48 bytes
     +------------------------+
     |     optional AHS 1     |\
     | - - - - - - - - - - -  | \
     |     optional AHS 2     |  \
     | - - - - - - - - - - -  |   > total length in AHS_length field in BHS
     |        . . . .         |  /
     | - - - - - - - - - - -  | /
     |     optional AHS n     |/
     +------------------------+
     | optional header digest | -- covers preceding (48 + AHS_length) bytes
     +------------------------+
     |                        |\
     |     optional data      | > total length in DATA_length field in BHS
     |                        |/
     +------------------------+
     |  optional data digest  | -- covers preceding (DATA_length) bytes
     +------------------------+


Thanks,

Bob Russell
InterOperability Lab
University of New Hampshire
rdr@iol.unh.edu
603-862-3774


On Tue, 13 Mar 2001 julian_satran@il.ibm.com wrote:


> 
> 
> Bob,
> 
> Interesting.  This is close to one of the variants I had for IETF-50 (a
> clear header).
> The only adavantage it has is that you are able to read all the AHSs in one
> read.
> The (admitedly academic) disadvantage it has is that you are limited in the
> size of extensions and have some redundancy.
> The basic issue that was raised - and there is no simple way out - is that
> once you have lost a block (BHS in your suggested layout) you are out of
> synch.  There is no simple way around it (at least not one that can be
> solved by changing layout) and an added digest (only the code or silicon to
> account for it) is IMHO not warranted by what you gain.  If you are willing
> to spend silicon or code on this the making header failures less probable
> and recovering if you fail by dropping the connection is a better bet
> (using the redundancy for a coding gain).  But this involves some
> complexity too.
> 
> Julo
>