Re: TCP RDMA option to accelerate NFS, CIFS, SCSI, etc.

[Erik Nordmark <Erik.Nordmark@eng.sun.com>]

> The TCP RDMA option reduces the overhead of receiving data over
> TCP-based protocols such as NFS and HTTP.  

Do you have any data (simulation, implementation) to back up this claim?
Or did you mean to say "provides a capbility which an implementation can
use to try to reduce the overhead"?
 
> It enables the construction of a simple hardware accelerator that
> copies data directly from the incoming packet into application
> buffers, avoiding expensive copies in the protocol stack.  Even
> without hardware acceleration, the option enables the protocol stack
> to decrease the number of copies it must do.

This seems to be an overstatement as well. Are you saying that an
implementation that currently has a single copy in its receive path
(from kernel to user space) can "reduce" the number of copies without
any hardware acceleration? That would imply that the number of
copies could be reduces to zero which I have a hard time understanding
(unless you add hardware acceleration).

> The TCP RDMA option is an annotation and requires no modifications to
> higher layer protocols. It can be used with popular protocols such as 
> HTTP, NFS, and CIFS, along with new protocols.

How will the higher layer protocol peer know what RDMA offsets to use?
For this to provide benefits to e.g. NFS it seems like you'd want the
NFS peer to have offsets that would depend on which file is being written.
Otherwise the only benefit would be for the NIC to be able to separate all
the protocol headers from the data so that the data can be placed on 
e.g. contigiuos pages in memory.

> The TCP option also provides a bit to indicate application-level
> message boundaries. The bit enables out-of-order processing of the TCP
> receive queue, potentially decreasing service times in the presence of
> packet drops and improving performance on parallel systems.
> 
> A draft describing the TCP RDMA option can be found at:
> ftp://ftpeng.cisco.com/pub/rdma/draft-csapuntz-tcprdma-00.txt

There is no DNS entry for ftpeng.cisco.com so I can't access the document.

Thanks,
   Erik