Re: Framing Discussion

[Michael Krause <krause@cup.hp.com>]

At 10:18 AM 12/22/2000 -0800, Mohan Parthasarathy wrote:

>It is a waste, if there is a better solution to tackle the same thing.
>If this can make things simpler, we should consider this as an option.
>But how does the sender insert this header ? Typically the target
>side would just dump all the data to TCP - which contains a iscsi
>header followed by data. TCP then would segment the data depending
>on the MSS. How do you insert such an header for each MSS sized
>data ?

This is a simple gather operation.  Most OS understand how to gather 
different segments into a single byte stream and also account for 
additional headers.  Nothing special here.


>The only RDMA proposal on Infinaband i have seen is the one from
>Microsoft. I am not sure which one you are referring to.

RDMA is defined within the InfiniBand architecture (within the workgroup I 
co-chair) - there is nothing for Microsoft to do here.  There have been a 
couple of drafts submitted to IETF on how to support RDMA over TCP which is 
what I'm referring to in this e-mail exchange.  The objective of a TCP RDMA 
solution should be to align as much of the semantics as possible so that 
bridging is simplified and products can easily inter-operate given the 
volume potential of iSCSI solutions relative to potential InfiniBand 
storage relative to InfiniBand servers in the long-run.


> > At the first BOF, I spoke about aligning the protocol with InfiniBand 
> since
> > that will eventually become the server point of attach in the coming
> > years.  The suggestion was made to include the same RDMA semantics (if 
> they
> > are supported) as InfiniBand.  It was further suggested there an in other
> > e-mail that a simple 8-byte header with a 4-byte CRC be associated with
> > each segment and that these fields be contained within the data payload so
> > that TCP is not impacted.  The contents of this header would contain a
> > 8-bit op-code, VA, length, etc. allowing the responder to target the 
> memory
>
>If VA is the virtual address, then this has security problems. How do
>you prevent arbitrary packets over-writing memory ? Actually all
>you need is a tag to identify the buffer pool, offset within a
>stream, and some smarts in the h/w to associate the tag to a
>buffer pool. Is this not sufficient ?

A VA is an address but what that address is a local issue and is only used 
as a key to find the real address.  As such, I don't see any security issue 
beyond that associated with any datea exchange on a network and the 
techniques even used today within FC products can be applied so the 
technology impacts are actually well understood and can be implemented by 
multiple vendors.

> > on the host / controller if all fields were valid.  If there was an
> > out-of-order delivery, the data could be spilled to temporary memory 
> either
> > in the host or the adapter and upon recovery, delivered to the correct
> > target buffer without requiring host processor intervention with a little
> > creativity.  This 12-bytes of overhead for SCSI operations would have
> > minimal impact on link utilization and overall solution efficiency.  I
> > believe these same concepts have been stated in the various RDMA proposals
> > that have been distributed and given the eventual movement to InfiniBand
> > for servers and the new SRP (SCSI RDMA Protocol), one might want to create
> > an iSCSI solution that can easily bridge into these other technologies.
> >
> > Note: The arguments about adapter complexity, impact to OS, etc. are 
> rather
> > moot in many ways.  The work will be done to support InfiniBand over the
> > next couple of years and thus the cost to implement / support is going to
> > be fairly minimal.  It should also be noted that many of these changes 
> have
> > already be done using PCI / PCI-X based solutions that support VIA,
> > Scheduled Transfer, Oracle, MPI, Sockets Direct, etc. so the ability to
> > deploy solutions in the highly desirable I/O interconnect independent way
> > is available today as well.  One does need to wait or rely upon InfiniBand
> > to make all of this happen.  It should also be noted that many companies
> > will be working to have Linux support for this type of technology in the
> > upcoming year so solutions should be available to all by the time iSCSI
> > ramps to volume in 2002.
> >
>I am not sure which proposal you are talking about. Infiniband working
>group is looking at the proposal from microsoft for RDMA. I think it
>is more specific to Infiniband.

I believe you are referring to the InfiniBand application workgroup which 
has been looking at how to bootstrap InfiniBand via SRP (SCSI RDMA 
Protocol) which is one possible mechanism to support storage over an 
InfiniBand fabric - there are several mechanisms being defined for 
boot.  This is a T10 not a Microsoft effort.  InfiniBand is simply trying 
to accelerate some of this work but T10 will own and drive it to 
completion. There is an upcoming T10 meeting Houston in January focused on 
SRP that people might want to attend concerning InfiniBand.

Mike