RE: IP Storage Framework document

["Bradley, Mark" <mark_bradley@btc.adaptec.com>]

Hi, David.  Thank you for your thoughts.  In answer to your
question, I think that:
1)TCP in h/w is probably possible, but as others have found, it
  is very difficult to do.  A simpler protocol in hardware is
  highly desirable
2)Cost.  TCP h/w implementations are likely to be complex silicon
  designs that one could expect are costly to provide.
3)While it is critical that IPS be cohabitable and non-destructive
  to network performance, it is entirely possible that a separate
  physical layer would be used for performance reasons.  I can
  envision many scenarios where we do this today, sans IPS.
4)Performance.  Both in CPU utilization and for pure bandwidth, it
  is desirable to go light to minimize impact for those who might
  not even implement in H/W.

Lastly, one envisions large networks in considering your points
below.  For small networks (SOHO, small business), your points
are likely less important for many reasons, including network 
loading.  Small networks with generall light traffic are less
sensitive to the issues you bring up.  They also tend to be more
cost sensitive.

These are the points that it I think you may not be sensitive to.
  --  markb 

-----Original Message-----
From: David R. Cheriton [mailto:cheriton@cisco.com]
Sent: Friday, February 04, 2000 12:06 AM
To: Dave_Lee@3com.com; Andreas Bechtolsheim
Cc: mark_bradley@btc.Adaptec.Com; Carl_Madison@3com.com;
Arun_Verma@3com.com; kquick@iphase.com; bassoon@ece.cmu.edu;
paulv@corp.adaptec.com; dwilson@corp.adaptec.com; raz@emc.com;
avb@cisco.com; rcampbel@cisco.com; Jgw@cisco.com; ddecapit@cisco.com;
sob@harvard.edu; vern@aciri.org; csapuntz@cisco.com
Subject: Re: IP Storage Framework document


I actually dont think there is any other transport protocol to
consider other than TCP.  Here's why.
1) TCP has been developed, refined, evaluated and deployed over a
   period of 20 years, with some of the smartest folks in the
   Internet research community spending years of effort.
   There is NO hope in duplicating this effort, IMHO,
   and plenty of evidence to suggest you'll screw up or end up
   with TCP with gratutious differences if you dont.
2) TCP has been the most successful transport protocol in the history
   of mankind, and continues to grow dramatically in use.
   There are numerous other attempts now and in the past to create
   a new transport protocol for one reason or another, but the
   success level has been almost 0 i.e. they're not all dead YET.
   (I was even responsible for one old attempt, blush!)
3) The TCP fast path is reasonably simple, has been identified
   in software implementations and used successfully for years.
   A hardware implementation of that fast path seems feasible to me,
   and, even if a little more difficult than what one might imagine
   for some new fangled protocol, the pay-off for a fast hardware
   TCP is certain, and huge - not so with any other protocol.

I think NFS/UDP is problematic.  Even in a campus network, you can
have systematic loss of IP fragments that UDP too often generates,
causing communication failures and network overload.
And, handling IP fragment reassembly is one of the more complex
aspects of hardware IP.  With a hardware TCP, I think one could
rely on MTU discovery and punt IP reassembly to software as
an unusual and unwarranted case.
Finally, my perception is that many netadmins are trying to reduce
and restrict non-TCP traffic on their networks because it does not
adapt predictably to network loads, causing failures.  A high-rate
UDP or random new protocol load on a network would not be welcomed.
I'd also mention that the same forces that are causing networks
to converge on IP are pushing for convergence on TCP as well.
Diversity is not nice in a mission-critical 7x24 enterprise network
that is struggling with rapid scaling in users, bandwidth, services.

So, the only thing that would convince me to consider another
transport protocol would be some major flaw in the above reasoning,
or proof that you just cant do hardware TCP at the required rates.
The latter would really, really surprise me.

What am I missing?

At 04:54 PM 2/3/00 -0800, Dave_Lee@3com.com wrote:
>
>
>
>Andy -
>
>You point out excellent issues that must be considered.  Let me provide
some
>additional comments for all to ponder.  Before I start, I believe the
document
>authors are using the term IP in its most generic sense and not meant to
imply a
>particular transport such as TCP, UDP, or some other transport.  Drew/Paul
--
>feel free to correct me if I'm wrong.  There are certainly many
possibilities
>and I think they all need to be considered at this point.
>
>While it may be theoretically possible to implement TCP in silicon, it is
not at
>all clear if TCP in silicon will occur anytime soon -- if at all.  Betting
on
>that fact may not sit well with others, especially those concerned with
higher
>performance campus-based solutions that they want available now.  Nothing,
>however, precludes us from taking advantage of TCP in silicon in a future
>revision of the protocol(s).
>
>For WAN traffic, I think most people would agree with you on your remark
on the
>need for bandwidth management.  Further, I believe most people would
probably
>agree with you that if we were looking for a pure L2 solution (e.g.,
directly
>attached LAN), the IETF is *not* the right place to take this.  I
definitely do.
>
>However, a customer may want to have a storage solution that can be
accessed
>over a campus network.  In this case, a L2 solution is not sufficient and
IP is
>required.  Further, it does not seem to be at all clear if TCP bandwidth
>management is a requirement in this environment.  The NFS version 4
requirements
>document has some commentary on this discontinuity.  In a campus network,
there
>are other possible ways to do bandwidth management, including traffic
>engineering, differential services, etc.
>
>In light of the NFS discussion, perhaps looking to NFS may not be a bad
thing to
>do?  NFS can operate over both TCP and UDP.  Perhaps this model is also a
good
>model for IP storage.
>
>Dave
>
>
>
>
>Andreas Bechtolsheim <avb@cisco.com> on 02/03/2000 09:14:54 AM
>
>Sent by:  Andreas Bechtolsheim <avb@cisco.com>
>
>
>To:   See Below
>cc:   See Below
>Subject:  Re: IP Storage Framework document
>
>
>
>While I will not be able to attend this meeting in person, I would
>like to propose an alternative framework for Storage over IP.
>We will discuss this in more detail at the 2/16 Meeting in San Jose.
>
>The key characteristics of our proposal are as follows:
>
>1. It is based on TCP/IP, so all the issues of a reliable
>   end-to-end byte stream including error recovery and
>   congestion management are addressed without inventing
>   a new protocol to do the same functions.
>
>2. It works over any network topology and any distance,
>   addressing the needs of both local and remote connectivity
>   between servers and storage devices.
>
>3. The protocol can be implemented efficiently in hardware
>   inside network interface controllers allowing for
>   high-performance wirespeed network operation.
>
>While in the past several new protocols have been proposed to
>accelerate certain types of data transfers, typically with
>the argument that they are easier to implement in hardware,
>experience has shown that such protocols have failed to gain
>market acceptance. With the plummenting cost of silicon,
>at this point in history the cost of implementing TCP/IP in
>NIC silicon is in the order of a few $ worth of silicon
>so there is simply no economic reason not to do it.
>
>In addition, proposing protocols that do not adhere to the
>TCP/IP bandwidth management algorithms pose a serious risk
>for overall network stability, in particular if they are used
>for high bandwidth traffic such as disk I/O.
>
>Finally, designing new protocols for limited topologies such
>as local area networks does not appear consistent with the
>goals of the IETF.
>
>
>
>To: Dave Lee/Hq/3Com               Raz @Emc.Com
>    Mark_Bradley @Btc.Adaptec.Com  Avb @Cisco.Com
>    Carl Madison/Us/3Com           Rcampbel @Cisco.Com
>    Arun Verma/Hq/3Com             Jgw @Cisco.Com
>    Kquick @Iphase.Com             Ddecapit @Cisco.Com
>    Bassoon @Ece.Cmu.Edu           Sob @Harvard.Edu
>    Paulv @Corp.Adaptec.Com        Vern @Aciri.Org
>    Dwilson @Corp.Adaptec.Com
>
>cc: Avb @Cisco.Com                 Csapuntz @Cisco.Com
>    Cheriton @Cisco.Com
>
>