Re: iSCSI:Can each do there own thing?

[Martins Krikis <mkrikis@yahoo.com>]

Let me try to analyze how I understand the possibilities.
Please note that I have included what I consider a compromise.
So, please read on.

Currently (12-94) there is no text explaining when a key=value pair is 
considered "sent". Thus, a side may consider it sent when it has sent 
the first byte of the key. The other side need not consider such a key
received (it only has the first byte) and may also send it. Both sides
feel like originators. Furthermore, each side has to be ready to receive
new key=value pairs after each PDU sent, so preparing more key=value
pairs than fit in one PDU is impractical. This prohibits the "batch-
processing" of keys, defined below.

One of the new proposals, we'll call it "no-origination on split-key scheme"
is such: we mandate that if one side receives a PDU ending with a 
"partial key" (start of key, but no '=') (note that this is not the same as
"partial key=value pair", which is defined by lack of terminating '\0'),
then it may not originate any new keys, but may send responses to the
keys (not necesarily key=value pairs) that it has received fully. 

There is a possible "twist" on this scheme, where it is also allowed to
originate new keys that are declarations.

The competing proposal, we'll call it "empty PDUs until end-of-data scheme"
is such: we mandate that until a node has seen the (new, to be added to
header) "data-end flag" from the other side (which signals that the
whole batch of key=value pairs that the other side wanted to negotiate
at the moment has been received), it MUST send only empty PDU-s (ones
with nothing in the DataSegment).

This proposal enables the "batch-processing" of keys, defined about
as follows:

    Prepare all key=value pairs you'd like to send;
    Send them with the peace of mind knowing that no text
    will come back until you've finished sending them completely;
    Receive all key=value pairs that the other side would like
    you to process together (sending only empty PDUs back meanwhile);
    Process all the received data.
    Repeat if necessary.

Note that the negotiation logic is PDU boundary agnostic, it is
only the sending layer that does something like that:

    if (have leftover) prepend leftover to current data
    chop off as much as fits in one PDU from the current data
    send the part just chopped off
    save the remainder in leftover

This, IMHO, is as simple as it gets. However, we ideally need a new
flag. While having the flag would be the cleanest, there is a way
to get by without it as follows (we'll call this "empty PDU on split-pair
scheme"): we mandate that if one side receives a PDU ending with a
partial key=value pair (i.e., not having '\0' at the end), it MUST
send an empty PDU. Thus, the presence or absence of the '\0' at the
end of the last key=value pair works as the "end-of-data" flag.

The batch-processing of keys approach works here as well, however, the 
sending layer needs to do a little more trickery. Namely, if there is more
data remaining to be sent in the subsequent PDUs, it must make sure
that the current PDU does NOT end in a '\0'. It needs to chop off
the '\0' if necessary and leave that for later as well. Padding makes
the task a tad more complicated, i.e., it may have to chop off (from
the tail end) some more bytes to have the PDU-end be properly aligned
and have no '\0' at the end, but it is possible.

The batch-processing approach does not work with the "no-origination
on split-key scheme", with the twist or without. If the last key and
'=' has fully fit in the PDU, the other side may start originating
keys, affecting what our side may have wanted to originate itself.
It isn't necessarily possible to force a key-split in the sending
layer, because the keys are of relatively short length, and in fact
(the remaining) data may consist of value alone. Thus, the sending
layer modification described in "empty PDUs on split-pair scheme"
does not work here. 

However, and here I come to the possibility of a compromise---we
can modify the "no-origination on split-key scheme" into a 
"no-originatio on split-pair scheme" as follows: 
we mandate that if one side receives a PDU ending with a 
"partial key=value pair" (no '\0' at the end)
then it may not originate any new keys, but may send responses to the
key=value pairs that it has received fully. Again, the twist allowing
to originate declarations is possible and functionally harmless.

If this were the case, then the following modified batch-processing
of keys approach would work:

    Prepare all key=value pairs you'd like to send;
    Send them with the peace of mind knowing that no keys
    (with the possible exception of harmless declarations)
    will be originated by the other side 
    until you've finished sending them completely;
    Meanwhile keep concatenating and buffering whatever the
    other side is sending back, but don't process yet.
    Receive any more data that the other side would like
    you to process at this point (sending only empty PDUs back meanwhile);
    Process all the received data.
    Repeat if necessary.

The sending layer needs the modification where it ensures that if
all data didn't fit, then the current PDU doesn't end with a '\0',
thus prohibiting the other side from originating keys.

This is IMHO very close to what Pat was proposing, yet does allow a variant
of "batch processing". It also allows a more agressive processing for
those who like it. This could be our compromise.

Summary:

1. "no-originating on split-key scheme"
2. "no-originating (except declarations) on split-key scheme"
3. "no-originating on split-pair scheme"
4. "no-originating (except declarations) on split-pair scheme"
5. "empty-PDUs until end-of-data scheme"
6. "empty-PDU on split-pair scheme"

2 is the least restrictive, however to take advantage of
the permission to originate declarations when negotiations may
not be originated, one has to look at key types. Thus, to take
advantage of this, it requires the most complicated implementations.
It is also longer to describe than 1. Otherwise, it has the same
properties as 1.

1 is what people seemed to be leaning towards yesterday.
This scheme does not allow "batch-processing" of keys, since the
sender has to be very PDU-boundary aware and leave the processing
of keys that don't fit for later. It utilizes the bandwidth better
than 5 or 6, but none of these schemes are considered for a very
likely or performace critical case(s).

The relation of 4 to 3 is the same as the one of 2 to 1.

3 is a modification of 1 where a "split PDU" is clarified as
not one with a split-key, but one with a split-pair at the end.
This enables the sending side to effectively force no-origination,
hence allowing modified but reasonably simple batch-processing of keys.
It also still allows more agressive modifications to not do batch
processing and send key responses sooner than a batch-processing
implementation would.

5 is the absolutely simplest scheme to implement and most restrictive
in terms of what the protocol allows. It welcomes batch-processing of keys
in its simplest form. Requires a new flag-bit in the PDU header
for Text/Login Request/Response PDUs (4 total).

6 is a modification of 5 that does not need the new flag in the
header. Still allows very simple batch processing; only the sending
layer needs a hack, allowing to force the other side to send only
empty PDUs.

Does this description seem accurate?

My order of preference for these schemes would be:

5, 6, 3, 4, 1, 2

Martins Krikis, Intel Corp.

Disclaimer: these opinions are mine and may not be those of my employer