RE: Wot I `know' about COWS in hardware

["Julian Satran" <Julian_Satran@il.ibm.com>]

Jim,

There are some things attractive about COWS -

1. the hard work - touching every data word has to be done only by the sender (on the normal path) and can be easily included in NIC with accelerator cards that seem to do a good job on the send side

2. If you are doing CRC or IPsec on a client in software there is no additional penalty (provided you can include the code in the right layer of software) as no data gets moved

3.It does not have to associated with a TCP packet alignment - and can work in face of TCP segmentation

Julo

"Jim Pinkerton" <jpink@microsoft.com> wrote on 17-12-2001 17:32:04:

> 
> My main concern with this approach is that we could kill the product but
> win the spec wars. Specifically, this approach means that an
> end-customer has one of two choices in deploying the technology:
> 
>    1) Upgrade both ends, and they'll see the full benefit
>    2) Upgrade only the server side, and see roughly 2-4 times the
> CPU
>       utilization on the client if their current 
>       implementation is optimized
>       on the client side (a mere 2x if they are doing
> significant
>       receives that already require a copy, more like 4x if
> they
>       are primarily doing sends, which currently has no bcopy
> in
>       many OS implementations).
> 
> This means that if they pick option 2) and their machines are CPU bound,
> that the data center capacity to handle requests will actually
> *decrease* if they deploy the technology. If the front end has enough
> idle CPU cycles, then they probably could select option 2).
> 
> In my experience, we need to make sure we have a volume solution to
> enable NIC vendors to make enough profit to fund the next generation
> (otherwise RDMA/TOE is a one-shot deal and won't keep up with the CPU
> architecture). This means we need a path to the front-end boxes in the
> data center. My concern is that there is no half-measure in the above
> scenario - the IT manager must upgrade the thousand front-end boxes at
> the same time as they upgrade the back end, or deploy asymmetric configs
> where some front end boxes are upgraded. I'm not sure how attractive
> this deployment scenario is.
> 
> Howard and Uri, can you comment on this issue?
> 
> 
> 
> Jim
> 
> 
> 
> 
> 
> 
> 
> > -----Original Message-----
> > From: Stephen Bailey [mailto:steph@cs.uchicago.edu]
> > Sent: Monday, December 17, 2001 7:13 AM
> > To: uri@broadcom.com; howard.c.herbert@intel.com
> > Cc: csapuntz@cisco.com; Jim Pinkerton; Julian_Satran@il.ibm.com;
> > allyn@cisco.com
> > Subject: Wot I `know' about COWS in hardware
> > 
> > Hi,
> > 
> > I haven't gotten a chance to do a full implementation yet, but here's
> > some architectural properties I believe to be true of a hardware COWS
> > implementation:
> > 
> >   1) can be implemented `in line' on receive
> >   2) requires an MTU-sized RAM on send
> >   3) expected touches to send RAM is 2 per data word (just the `fifo'
> >      write and read ops, no editing), assuming the headers are merged
> >      on the outbound side.
> >   4) worst case touches to send RAM is 3 per data word (assuming every
> >      word must be edited)
> >   5) eliminates the need for the funky `make sure you don't send
> >      anything that's false positive under non-nominal conditions'
> >      behavior of the key/length proposal (I kinda doubted hardware
> >      impls were going to do this anyway, since it was a SHOULD).
> > 
> > Basically, it looks OK to me.  Slowing the sender is much better than
> > slowing the receiver.  Theoretically, we could reverse the pointer
> > chain and allow in-line send, but RAM on receive, but that seems
> > stupid to me.
> > 
> > It's clearly a design tradeoff whether you chose to use the COWS
> > send-side RAM for other purposes, or not.
> > 
> > I'm hoping you guys can tell me whether you think this blows your
> > budget, or other noteworthy (unfortunate) properties.  As you can
> > tell, I have the utmost enthusiasm for the mission (Dr. Chandra...).
> > 
> > Thanks,
> >   Steph