I
think the COBS/COWS should more potential than the markers proposal,
or
the ULP framing without COBS. There is one case
where it does add more overhead,
but the question is how prevelent is the scenario -
when outbound zero copy is
enabled/possible and the NIC does checksum offload
and cannot be changed to
do
COBS. Of course changes are required :-).
I
also hope that data-centers will use acclerated iSCSI/Clustering HBAs/NICs
rather
than the current solutions. The current solutions
MAY be useful for desktops/laptops
where hopefully there are plenty of spare cycles to
do COBS in software.
COBS also has alignment benefits - the header could
be aligned with the ULP PDU,
and the ULP PDU can be aligned with the TCP header
and there are no false
positives. The alignment with the TCP header may
not always happen (the mythical
box in the middle that does TCP resegmentation),
but can be detected - in the
presence of such a box, the performance could
reduce to the levels encountered
when IP fragmentation happens.
I
think it is better to have a couple of inter-operable implementations that
demonstrate
the benefit of any of the alternate proposals
(especially markers vs cobs) before selecting
one.
Jim,
There are some things attractive
about COWS -
1. the hard work
- touching every data word has to be done only by the sender (on the
normal path) and can be easily included in NIC with accelerator cards that
seem to do a good job on the send side
2. If you are doing CRC or IPsec on a client in
software there is no additional penalty (provided you can include the code
in the right layer of software) as no data gets moved
3.It does not have to associated with a TCP packet
alignment - and can work in face of TCP segmentation
Julo
"Jim Pinkerton" <jpink@microsoft.com> wrote on 17-12-2001
17:32:04:
>
> My main concern with this approach is that
we could kill the product but
> win the spec wars. Specifically,
this approach means that an
> end-customer has one of two choices in
deploying the technology:
>
> 1) Upgrade both
ends, and they'll see the full benefit
> 2) Upgrade
only the server side, and see roughly 2-4 times the
> CPU
>
utilization on the client if their current
>
implementation is optimized
>
on the client side (a mere 2x if they are doing
>
significant
> receives that already require a
copy, more like 4x if
> they
> are
primarily doing sends, which currently has no bcopy
> in
>
many OS implementations).
>
> This means
that if they pick option 2) and their machines are CPU bound,
> that
the data center capacity to handle requests will actually
>
*decrease* if they deploy the technology. If the front end has
enough
> idle CPU cycles, then they probably could select option
2).
>
> In my experience, we need to make sure we have a
volume solution to
> enable NIC vendors to make enough profit to
fund the next generation
> (otherwise RDMA/TOE is a one-shot deal
and won't keep up with the CPU
> architecture). This means we need a
path to the front-end boxes in the
> data center. My concern is that
there is no half-measure in the above
> scenario - the IT manager
must upgrade the thousand front-end boxes at
> the same time as they
upgrade the back end, or deploy asymmetric configs
> where some
front end boxes are upgraded. I'm not sure how attractive
> this
deployment scenario is.
>
> Howard and Uri, can you comment
on this issue?
>
>
>
> Jim
>
>
>
>
>
>
>
> > -----Original
Message-----
> > From: Stephen Bailey
[mailto:steph@cs.uchicago.edu]
> > Sent: Monday, December 17,
2001 7:13 AM
> > To: uri@broadcom.com;
howard.c.herbert@intel.com
> > Cc: csapuntz@cisco.com; Jim
Pinkerton; Julian_Satran@il.ibm.com;
> > allyn@cisco.com
>
> Subject: Wot I `know' about COWS in hardware
> >
>
> Hi,
> >
> > I haven't gotten a chance to do a full
implementation yet, but here's
> > some architectural properties
I believe to be true of a hardware COWS
> >
implementation:
> >
> > 1) can be implemented
`in line' on receive
> > 2) requires an MTU-sized RAM on
send
> > 3) expected touches to send RAM is 2 per data
word (just the `fifo'
> > write and read ops,
no editing), assuming the headers are merged
> >
on the outbound side.
> > 4) worst case touches to
send RAM is 3 per data word (assuming every
> >
word must be edited)
> > 5) eliminates the need for
the funky `make sure you don't send
> >
anything that's false positive under non-nominal conditions'
>
> behavior of the key/length proposal (I kinda
doubted hardware
> > impls were going to do
this anyway, since it was a SHOULD).
> >
> > Basically,
it looks OK to me. Slowing the sender is much better than
>
> slowing the receiver. Theoretically, we could reverse the
pointer
> > chain and allow in-line send, but RAM on receive, but
that seems
> > stupid to me.
> >
> > It's
clearly a design tradeoff whether you chose to use the COWS
> >
send-side RAM for other purposes, or not.
> >
> > I'm
hoping you guys can tell me whether you think this blows your
> >
budget, or other noteworthy (unfortunate) properties. As you
can
> > tell, I have the utmost enthusiasm for the mission (Dr.
Chandra...).
> >
> > Thanks,
> >
Steph
