Re: TCP RDMA option to accelerate NFS, CIFS, SCSI, etc.

[Zachary Amsden <zamsden@cthulhu.engr.sgi.com>]

>    From: Zachary Amsden <zamsden@cthulhu.engr.sgi.com>
>    Date: Thu, 24 Feb 2000 17:56:26 -0800
> 
>    With network bandwidth approaching memory bus bandwidth, it becomes
>    an issue.  A single copy receive uses 3x the bus bandwidth of a
>    zero copy implementation (or worse if not aligned properly), and
>    also poisons many more cachelines.

> In my experience cpu cycles are abundant on client machines.  Server
> side is where getting precious cpu cycles back seems more important.

Agreed for the most part.  But reducing latency and cache misses are just as 
important on servers, and it would be interesting to see whether RDMA would 
make a difference there.  Of course for file/web serving, the server receive 
fast path can all be done in-kernel anyways, and output volume dwarfs input 
volume.

I can come up with a rigged scenario where RDMA should make a difference:

Group server or database which needs large amounts of external data over the 
network via NFS/your protocol of choice.  Internet search engines would 
possibly fall into this category.  In this case, input/output ratio is 
drastically different than a typical server, and memory bandwidth used by 
copying could become an issue.

However, this is a pretty limited use, and would only need to be deployed on a 
few machines.

I don't see RDMA being beneficial for typical web/file servers, or clients, 
other than getting a couple % extra performance.

-- 
Zachary Amsden  zamsden@engr.sgi.com  (650) 933-6919  09U-510  Core Protocols