RE: [Tsvwg] RE: iSCSI: Framing Steps

["Douglas Otis" <dotis@sanlight.net>]

Shridhar, David,

>Shridhar Mukund wrote:
>>A few more clarifications:
>>
>>(1) If the data-dependent memory allocator below TCP gets confused,
>>    that's its problem, and iSCSI's problem, not TCP's.  As long
>> 	as the inbound data is presented to TCP in the order it came
>> 	off the wire, TCP's processing behavior is unchanged.  If the
>> 	data is delivered by TCP in locations other than those iSCSI
>> 	was expecting, it can be copied to the right locations with
>>    suitable safeguards.

Let's say the packet received off of the wire is not in contagious memory
(as typical) but is a memory array.  The layer below TCP constructs this
array to present information in the correct location as per the iSCSI
structures found within the content of the expected TCP byte stream.  This
assignment process will require prior state to be considered in performing
this assignment, in addition to sharing the identification of these
locations with the iSCSI application.  Should there be an error in this
process, the normal expectations of correct placement is no longer valid as
a result.  As there must already be communication between the application
and this "below TCP layer process", posting error information seems like a
small detail.  Indeed, one could assume the application would track each
piece of information in this array to detect an error, but this would not
likely the solution used as the intent is to minimize overhead.  I would
hope that if this scheme is to be used, a definition of the information
relayed between the application and the application specific "below TCP
layer process" would be included within the description of this technique.

The safeguards would need to include that this layer acts to prevent
duplications to avoid overwritting information that may be already found in
a prior array sent to TCP.  Should TCP processing of these packets not match
exactly with this "below TCP layer process", there would be potential for
these differences to create problems.  In addition to matching the view of
TCP within this "below TCP layer process", errors found in the processing of
the iSCSI structures within the "below TCP layer process" will also need
special handling.  The simple act of placing the iSCSI data into the correct
location based on the content of the TCP byte stream implies that the "below
TCP layer process" understand beforehand this byte stream.

>>(2) FIM processing makes no changes to packet contents, period.
>>	I have no idea where Doug got the impression "it is not the packet
>> 	location that is being changed, but the content of the packet
>> 	or it would be of little value".  That is simply not true -
>> 	the FIM markers can't be removed below TCP because they were
>> 	sent in the TCP bytestream and have to be presented to TCP.
>> 	The contents of each packet may not be contiguous in
>> 	memory, but that has been common in TCP implementations
>> 	from the beginning (e.g., see the original BSD mbuf code).

It is not a reassignment of the packet, but detailed placement of the
content within the packet that I was referencing.  This operation is not
done on the packet as a whole, but to many portions of information found
within the packet.  This operation is not a simple, linear, sequential
placement of information.  An array could make such appear as a contagious
packet following this operation.  I agree with that point.  I was attempting
to make clear the function that was being performed and the level of
complexity involved.

>>(3) Zero-copy TCP is irrelevant to this discussion.  In an integrated
>> 	TCP/FIM/iSCSI implementation, there would be no address boundaries
>> 	between any of the components and hence no need to avoid the copy
>> 	across address boundaries that is the motivation for zero-copy TCP.
>>	Zero copy TCP schemes cannot achieve "appropriate packet placement"
>> 	for iSCSI by themselves because they're not capable of ensuring that
>> 	a 4k SCSI data payload lands in 4k of contiguous memory aligned to
>> 	a 4k boundary, and hence will cause copies to be made.

The description that David was using sounded as if he was describing Zero
Copy TCP.  I felt there was detail lacking in this description.  iSCSI
structures would not ensure page alignment, and the process of placing and
splicing together prior information would be part of this "below TCP layer
process."  In addition, there would be a requirement to hold some packets
should information arrive not aligned within the TCP segment or before the
marker.  This additional requirement together with a more complex state
becomes motivation for transitioning this scheme toward creating a framed
version of TCP.  My argument from the very beginning was that we should not
consider changing TCP and that SCTP provided the needed framing as well as
removed the need for placing the application below the transport.  This
debate has evolved into semantics as to what constitutes a change to TCP.
Placing the application below TCP seems like a change.

David Black wrote:
>
> Beyond this, Doug continues to claim that there are new security
> issues in FIM but has failed to describe any significant new risks,
> aside from a general concern that implementation work could introduce
> security bugs - that is the case for most IETF protocols, and is not
> a reason to stop working on protocols.

Perhaps a full description of the "below TCP layer process" would be
helpful.  Examining the state created, the types of commutations relayed
between this layer and the application would be needed for assessment.  If
the goal is to allow applications a common service or API as was once
possible with TCP, then these definitions are important and to allow risks
to be assessed.  My added concern was that using an interval that was a
power of 2 would allow a prediction of a where the mark would be found in
the future.  I think it would be incumbent to present full details of this
scheme to allow a complete review and for a standardize interface to emerge.
The state found in this "below TCP layer process" in conjunction with the
types of application communications are potential areas where security may
become problematic.

Doug