Re: TCP RDMA option to accelerate NFS, CIFS, SCSI, etc.

[Costa Sapuntzakis <csapuntz@cisco.com>]

Hi,

Your input on this is very much appreciated. I'd like to clarify a couple
things, though. Some of these things may not need clarification. :)

The TCP RDMA option is not about accelerating sending of data. It's
about speeding up the receiving of bulk data.

The TCP RDMA option isn't proposing to only about accelerating servers.
It's about accelerating the receiver of data. In the case of NFS READ
RPCs, that's the NFS client.

I agree that for specific problem domains, SCSI, NFS, HTTP, people may
want to build specialized server hardware.

Perhaps explaining why I decided to explore this space will help.
Today, you have specialized silicon that for simple bus protocols
(SCSI parallel interface and ATA) will directly take transfer blocks
between the device and the buffer cache. This is not currently done
with TCP, to the best of my knowledge. The best TCP implementations
do zero copy to a TCP receive buffer.

However, in the case of most storage protocols, you don't want
the data in the receive buffer. You want it in the buffer cache, so
there is a copy to the buffer cache.

So, NFS has a  CPU overhead hit as compared to optimized storage host bus
adapters. The goal was to eliminate part of this hit, by getting rid of an
extra copy.

Now, this proposal doesn't fix the interrupt overhead problem. 
Optimized FC/SCSI NICs have one interrupt/transfer or less.

-Costa

On Thu, 24 Feb 2000, David S. Miller wrote:

>    Date: Thu, 24 Feb 2000 14:59:38 -0800 (PST)
>    From: Erik Nordmark <Erik.Nordmark@Eng.Sun.COM>
> 
> As an aside I think the RDMA proposal has a lot of holes too.  For
> example, there are in-kernel HTTP accelerators that do the complete
> client header parse and initial packet response in the hw interrupt
> handler.  There are no user buffers involved, and static response
> data is DMA'd directly from the filesystem page cache.
> 
>    > A draft describing the TCP RDMA option can be found at:
>    > ftp://ftpeng.cisco.com/pub/rdma/draft-csapuntz-tcprdma-00.txt
> 
>    There is no DNS entry for ftpeng.cisco.com so I can't access the
>    document.
> 
> Here is what I get:
> 
> ? host -a ftpeng.cisco.com
> Trying null domain
> rcode = 0 (Success), ancount=1
> The following answer is not authoritative:
> The following answer is not verified as authentic by the server:
> ftpeng.cisco.com        84574 IN        CNAME   ftp-eng.cisco.com
> For authoritative answers, see:
> cisco.com       38435 IN        NS      NS1.cisco.com
> cisco.com       38435 IN        NS      NS2.cisco.com
> Additional information:
> NS1.cisco.com   78995 IN        A       192.31.7.92
> NS2.cisco.com   67536 IN        A       192.135.250.69
> rcode = 0 (Success), ancount=1
> The following answer is not authoritative:
> The following answer is not verified as authentic by the server:
> ftp-eng.cisco.com       84574 IN        A       198.92.30.33
> For authoritative answers, see:
> CISCO.com       38435 IN        NS      NS1.CISCO.com
> CISCO.com       38435 IN        NS      NS2.CISCO.com
> Additional information:
> NS1.CISCO.com   78995 IN        A       192.31.7.92
> NS2.CISCO.com   67536 IN        A       192.135.250.69
> 
> 
>