Re: iSCSI: Towards Urgent Pointer Consensus

["Venkat Rangan" <venkat@rhapsodynetworks.com>]

Jim Williams wrote:

> I would like to outline my understanding of how the
> urgent pointer framing mechanism would work.  This
> is done with the hope the any misunderstandings
> will be corrected and any gaps filled.
> 
> My understanding is that iSCSI MUST be fully interoperable
> with all existing TCP implementations and MAY contain
> features that allow customized TCP implementations to
> extract additional performance.  Use of the urgent
> pointer to delineate message boundaries in the 
> presence of dropped packets is such a feature.
> 
> This creates four cases that should be considered.
> 
> 1.  standard TCP sending to standard TCP
> 2.  customized TCP sending to standard TCP
> 3.  standard TCP sending to customized TCP
> 4.  customized TCP sending to customized TCP
> 
> It seems clear that in case 1 and 2, urgent data
> should not be used.  There is no way for a standard
> TCP to use received urgent data designations to
> any advantage, and it would only slow things down.
> 
> Case 4 can easily insert the urgent markers where
> needed and extract them to optimum advantage on
> the receiving end.  The only potential problem
> with this case is if the urgent pointer is
> modified within the network (or more accurately
> by some gateway in the network path).  If the urgent
> pointer is moved to point to some byte NOT indicated
> as urgent by the sender, then the scheme is broken.
> We NEED to guarantee that this can't happen.
> If the urgent pointer is coalesced so that some
> bytes designated as urgent at the sender are no
> longer urgent when received by the receiver,
> everything should still work, but possibly with
> lower performance.  If this happens rarely, 
> no problem.  If it happens often, then perhaps
> the urgent data mechanism is not the best way
> to recover framing.
> 
> Case 3 is the most problematic.  We are potentially
> lowering the performance on one end in order to 
> improve the performance on the other.  Case 3
> has the same concerns as case 4 with the additional
> concern that the sender may coalesce the urgent
> pointer in some cases.  These concerns can
> be resolved in a number of ways:
> 
>   a. Negotiate the use of urgent data only in case 4.
> 
>   b. Negotiate the use of urgent data in cases 3 and 4,
>      and accept the fact that the performance gain in
>      case 3 may in some cases be small due to excessive
>      coalescing of the urgent pointer.
> 
>   c. Verify that for most TCP implementations, sent
>      urgent data is usually received at the remote
>      end with the TCP urgent marker intact.  In this
>      case everything is fine (at least from the 
>      receiver's view point).
> 
> 

In cases 3) and 4) also, we do not need Urgent Pointer, because
the bandwidth achieved will drop to a point (in response to packet
drops), where you do not need a large amount of anonymous buffers.
Additionally, the receiving end in both cases is a custom TCP
implementation. It is possible to design such implementations
where anonymously held buffers get "transferred" to its rightful
owner when sequence holes get filled, without any additional data
copies. As long as the TOE memory address space is also addressable
by higher-level (iSCSI) software layer, this copy can be avoided.
Perhaps, your "custom implementation" assumes certain constraints
on memory addressability between iSCSI and TCP layers.

Venkat Rangan
Rhapsody Networks Inc.
www.rhapsodynetworks.com