RE: RDMA over TCP (Was Re: VI (Was: Avoiding deadlock in iSCSI))

["Douglas Otis" <dotis@sanlight.net>]

Jim,

In summary, you expect kernel modifications of TCP API to allow frame
alignment, a hardware based 32-bit CRC that uses a different polynomial than
FCP and a direct copy method at the NIC.  As SCSI does not benefit from VI
with zero copy already possible with frame alignment, the area where there
could be commonality has been missed with the choice of CRC.  Rather than
modifying thousands of TCP implementations, it would seem appropriate to
leave TCP as is and concentrate on providing these improved features using
SCTP designed to handle these requirements together with additional features
yet to be resolved with TCP.  You need not convince systems to change TCP to
benefit from rather important features found with SCTP.  Do not encumber TCP
with disruptive options.  SCTP can emerge using UDP and become supported
without making these changes to TCP that you advocate as evolutionary.  Once
you implement frame alignment, the API for TCP becomes unusable as it does
not afford means of discerning frames.  SCTP implementations already address
these problems and so I do not see your approach as evolutionary but rather
revolutionary.  You do not get my vote on this approach.

Doug


> -----Original Message-----
> From: Jim Williams [mailto:jimw@giganet.com]
> Sent: Tuesday, September 26, 2000 10:00 AM
> To: Douglas Otis; ips@ece.cmu.edu
> Subject: Re: RDMA over TCP (Was Re: VI (Was: Avoiding deadlock in
> iSCSI))
>
>
>
> -----Original Message-----
> From: Douglas Otis <dotis@sanlight.net>
> To: Jim Williams <jimw@giganet.com>; ips@ece.cmu.edu <ips@ece.cmu.edu>
> Date: Monday, September 25, 2000 5:56 PM
> Subject: RE: RDMA over TCP (Was Re: VI (Was: Avoiding deadlock in iSCSI))
>
>
> >You suggest a frame aligned formatting scheme is possible using an
> >unmodified TCP.
>
> Yes.
>
> >By allowing out of sequence delivery of TCP, are you opening the
> window for
> >spoofing?  One need not guess the sequence, just get in front of
> it a bit.
> >After that, you will never recover state.
>
> A 64 bit randomly selected connection ID should prevent blind spoofing.
> For an attacker who snoops valid packets and then spoofs, we would
> be vulnerable.  However no window is opened, just an already open window
> failed to close tight.
>
> > Do you expect the RDMA option to be universally supported, and if not?
>
> The proposal is not an option, it is a format.  The format could
> potentially
> be shared by multiple protocols.  It is by no means universal; it
> applies only to those protocols that adopt it.
>
> >Do you see this level of CRC done in software?
>
> I see it being done in hardware in a NIC by anyone who cares about
> performance.  That's largely the case for TCP checksums now.  Non
> performance critical applications could do it in software.
>
> >> It is
> >> RECOMMENDED that each TCP segment contain exactly one RDMA segment.
> >
> >How do you achieve this recommendation?
>
> With cooperation of the TCP implementation.  Without that, it cannot
> in general be guaranteed.  Typically for performance critical
> applications, NICs will handle both TCP and RDMA protocol processing,
> and in this case it is easy.
>
> >Sending 4GB exchanges, you still have queue blocking if not head of queue
> >blocking.
>
> Yes.  If that's a problem, don't send 4GB messages.  All I am saying
> is that the format allows 4GB messages.
>
>
> >Can one suggest frame alignment within TCP?
>
> Certainly one can suggest it.  Will the suggestion be supported?
> Does it violate any rules that can't be changed?  Perhaps I
> will find out.
>
> >Have you investigated the use
> >of SCTP?  SCTP does solve all the related problems without violating
> >protocol or protocol API.
>
> SCTP is an interesting protocol with a lot of merrit and a lot of
> good ideas.  But my guess is that the market wants one transport
> protocol and wants it to be TCP.  My vote would be to stick with
> TCP and evolve it in a way that meets the requirements of the
> market and is fully compatable with the existing infrastructure.
>
>