RE: Why FCP doesn't need RDMA? It has a better way.

["Y P Cheng" <ycheng@advansys.com>]

> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Black_David@emc.com
>
> With my WG co-chair hat off:
>
> From an iSCSI viewpoint, I don't see iSCSI by itself as being sufficient
> to motivate a protocol-independent RDMA - an iSCSI HBA could understand
> the iSCSI headers and interact with DMA in the same fashion as existing
> HBAs.  The task before those interested in RDMA is to identify a set
> of protocols for which a common RDMA mechanism makes sense from
> an engineering standpoint.  I tend to agree with the previous emails
> that iSCSI could make optional use of a common RDMA mechanism
> if available, but must not REQUIRE its use.

Yes, we must have a common DMA/RDMA mechanism for iSCSI whether the iSCSI
PDU travels on Ethernet, 1394, fibre channel, or InifiniBand.  (FYI, there
are TCP/IP and SCSI implementations on 1394).  For most adapters, not all,
they get the DMA destination address from the device driver.  The incoming
data frames only specifies an offset from the destination.  The exchange ID
or SCSI tag identify a request, hence, the DMA destination.  For VI RDMA the
initiator sends a memory handle in the FC-VI header to the target, which, in
turn, passes the handle back together with an offset to read or write the
initiator's memory.  For 1394 the target gets an ORB, Operation Request
Block, from the initiator.  In the ORB, a scatter and gather list of the
initiator's memory is provided.  The target works through the scatter/gather
list using a "physical read or write" protocol unique to 1394.  This lack of
a commonality puts a lot burden on the adapters to deal with different
protocols. IP, FCP, and VI all have different locations for their offsets
and not everyone use the fibre channel exchange ID.  I have not read the
TCP/RDMA proposal, but, something like the ORB of 1394 would be nice.  This
gives total control to a target with minimum interference to an initiator.
We should have a mechanism in iSCSI to allow some adapters to execute
100,000 IOs per second.  Of course, not all adapters will be required to
support DMA/RDMA.  Keep in mind, while God forbids, some iSCSI
implementation might not use TCP/IP or SCTP. I hope, this does not violate
the charter the WG.  Yes, this DMA/RDMA discussion can take months.   :-)

Y.P. Cheng, CTO, ConnectCom Solutions Corp.