Re: TCP RDMA option to accelerate NFS, CIFS, SCSI, etc.

["Justin T. Gibbs" <gibbs@FreeBSD.org>]

>> The performance impact of RDMA is quite a bit larger than SACK, so
>> I don't know that your example is relevant.  All the big vendors
>> implement zero-copy in some shape or form and since RDMA is a scheme
>> to make zero-copy work in more cases, I'm sure it will be picked up
>> if the proposal is deemed sane.
>
>Can you elaborate on this?  Suppose TCP "blindly" does zero copy everything to
>an app's buffer (for example, to a web browser's receive buffer) without
>RDMA.  Then the browser app looks at the data and displays it.  What is the
>difference RDMA makes in this case?  Yes, RDMA can separate different messages
>in the buffer.  But this can also be done by the browser app, not by TCP.

You seem to be saying that in the common case zero copy is achievable.
Most implementations I've seen require the network driver to make
a guess about where the payload will be in an incoming packet so the header
can be stripped off and the payload dmaed to an aligned area.   A page
flip is then performed to get the data where the user wants it,
imposing the restriction that your  payload be page sized so you don't
leave gaps in the user's destination buffer.  Certainly, with a more
intelligent network adapter that knows every protocol you can determine
exactly where the data is in each packet.  If you add connection tracking
and sequence number sniffing to the nic with a mechanism to register user
buffers to connections, you can get zero copy every time*.  Unfortunately
this is not very general purpose solution.  The point of RDMA seems to be
to allow nic manufacturers to add support for a single tcp option that, at
the very least, allows the nic to align the payload for you.  Add RID
registration with the nic and you get the payload exactly where you want it
too.  All without too much state information kept by the nic.

* This technique has been implemented with custom firmware on  Alteon
  Gig-E cards for a product I work on.
  single protocol
--
Justin