Re: TCP limitations (was Re: ISCSI: Urgent Flag requirement violates TCP.)

["Randall R. Stewart" <randall@stewart.chicago.il.us>]

Jim:

Ok, I will review this draft with the framing problem in mind...

Here are the list of my issues (on the fly as I read it this
A.M.... better go get my first cup of coffee ;->)


A) Section 3.3 has a note:
"
Note that RDMA_Read_Request messages always consist of exactly one 
segment and contain no payload data.
"

You can NOT make this statement without a modified TCP to make it
happen. Depending on congestion and other factors TCP will bundle
things together no matter if you set the PUSH bit, the URGENT bit
or whatever... I think you need to remove this "Note"

B) Section 3.4 for the "order field" has this description

"
This 16 bit field defines the ordering requirements on a message.  A 
value of "N" in the order field indicates that the payload data may not 
be read from or written to its ultimate source or destination until all 
EXCEPT the preceding N messages have been processed.  The value of 
0xFFFF (all one bits) is reserved to indicate that the operation may be 
done immediately when received unconditionally.
"

Saying that "data may not be read from or written to its ultimate..."
bothers
me. There is a subtle un-intended idea here that you can't read or write
until... I realize that the "ultimate destination" is supposed to point
out that yes, you can read it to a temporary buffer, but I would think a
rewording
to make it more clear here would be better... say
replacing the "payload data may not be read from or written..." with
just 
...the payload may not be processed until all EXCEPT the preceding N...


C) Section 5, the real method of the "RDMA" framing is in
   this paragraph.

"
The recommended method for doing this is to assume that the RDMA 
segment is aligned with a TCP segment, and verify the correctness of 
the header fields of the RDMA segment.  If these header fields are not 
correct, then the NIC should fall back to buffering the packet until it 
can be processed in order.  In particular, the 64 bit connection ID 
field was selected at random, so the only way that could match payload 
data is by pure chance.  It can be easily shown that even if a
miss-aligned packet arrives every 2us, the MTBF of mistakenly 
identifying this as an aligned packet is greater than one million 
years.  Checking the message number, CRC, and other fields only 
enhances the confidence in this determination.
"

1) You recommend that you assume the header is aligned with a
   TCP segment.. This really can NOT be made to happen without
   hacking TCP, but I can see where you might want to make a
   "guess at it". One may want to tweak this wording a bit... I
   need more coffee to think on it :)

2) When you miss a segment, you start into the ole digging through
   the byte stream looking for a "magic sequence" that is not escaped.
   If you find it you "verify the header" and off we go... Now this may
   work, but I am not sure about the processing power needed to dig
   through every byte that is streaming in while still receiving and
   buffering data... This may be ok, others more familiar with what
   can be done on a card will have to comment on this....


Basically you have chosen the "magic sequence" method that I believe
was proposed earlier on the list.

Regards

R






Jim Williams wrote:
> 
> ----- Original Message -----
> From: Matt Wakeley <matt_wakeley@agilent.com>
> To: <ips@ece.cmu.edu>
> Sent: Monday, November 27, 2000 3:35 PM
> Subject: Re: TCP limitations (was Re: ISCSI: Urgent Flag requirement
> violates TCP.)
> 
> > csapuntz@cisco.com wrote:
> >
> > > Given that shim protocol requires no changes to the TCP in the sender,
> > > it is currently my favorite way of doing RDMA.
> > >
> > > -Costa
> >
> > Please see Jeff's message on 10/26.  If you don't have framing, when you
> lose
> > a segment, you need to buffer generic TCP segments on the NIC before you
> can
> > "RDMA" them somewhere, because you don't know where the "RDMA" information
> is
> > in the "byte stream".
> >
> > -Matt
> >
> 
> http://www.ietf.org/internet-drafts/draft-williams-rdmatcp-00.txt
> 
> is an example of a shim layer that provides a mechanism
> to recover framing upon loss of TCP segments.

-- 
Randall R. Stewart
randall@stewart.chicago.il.us or rrs@cisco.com
815-342-5222 (cell) 815-477-2127 (work)