RE: IETF mailing list question on Storage over Ethernet/IP

["HAAGENS,RANDY (HP-Roseville,ex1)" <randy_haagens@hp.com>]

David,

A few comments about this one.

1. FC does not provide reliable transmission.  It provides for error
detection, but escalates recovery to "upper level protocol".  FCP-2 has
improved this situation, but is not widely implemented yet.  One of the
advantages of using a transport such as TCP is that link errors will be
corrected in a manner that is transparent to the application protocol
(SCSI).

2. Jumbo frames will not be necessary when TCP is implemented in hardware.
Most FC implementations use 1024 byte frames, and performance is very
adequate, given hardware implementation of FCP.

3. The cost of using different transport protocols in the LAN and WAN is
that the two will not interoperate.  Many of us believe that TCP has proven
itself in both the LAN and WAN.  I bet your PC or UN*X workstation is using
TCP for all its protocol needs.

4. The IPS working group is mapping SCSI to TCP.  Another working group is
mapping FC to IP.  These are very different approaches.  The first (ours)
preserves SCSI, but does not include any vestige of Fibre Channel.  It is
intended for use in the LAN, MAN and WAN.  Its best use is for connecting
hosts computers to storage controllers using Ethernet and IP WAN technology.
It will be possible, but non-trivial, to translate between SCSI over TCP/IP
and SCSI over Fibrechannel.  The second is a tunneling scheme for extending
Fibre Channel over the IP WAN.  It does not contemplate Ethernet-based hosts
or storage controllers.

5. Just about any reliable transport will do nicely for transporting SCSI
commands.  We chose TCP because its implementation and behavior are
well-known, and it is well-supported with load-balancing, QoS and security
features.  While another protocol (such as reliable datagram) might be
arguably better suited to storage transport applications, we'll use TCP
"because it's there".  We'll have the benefit of all the other investment
that's going into improving TCP for internet uses.

Randy Haagens
Networked Storage Architecture
Storage Organization
Hewlett-Packard Co.
e-mail: Randy_Haagens@hp.com
tel: +1 916 785 4578
fax: +1 916 785 1911


> -----Original Message-----
> From: Dave Nagle [mailto:bassoon@yogi.ece.cmu.edu]
> Sent: Thursday, May 25, 2000 4:29 PM
> To: SCSI-over-TCP List
> Subject: IETF mailing list question on Storage over Ethernet/IP 
> 
> 
> 
> ------- Forwarded Message
> 
> Date:    Thu, 25 May 2000 19:27:02 -0400
> From:    Dave Nagle <bassoon@yogi.ece.cmu.edu>
> To:      "Jon William Toigo" <jtoigo@IntNet.net>
> cc:      ietf@ietf.org
> Subject: Re: Storage over Ethernet/IP 
> 
> Jon,
> 
> Original Message
> - ----------------
>  >> I am seeking a few points of clarification:
>  >> 
>  >> 1.  Fibre Channel folks have attempted to explain to me 
> why TCP/IP could =
>  >> NEVER be a viable interconnect for block level storage 
> operations.  They =
>  >> claim:
>  >> a.  TCP is too CPU intensive and creates too much latency 
> for storage =
>  >> I/O operations.
>  >> b.  The IP stack is too top heavy and processing packet 
> headers is too =
>  >> slow to support storage I/O operations.
> 
>   There is a lot of work to show that this is not true.  Check out Van
> Meter's 1998 ASPLOS paper "VISA - Netstations virtual internet SCSI
> adaptor."
> 
>  Perhaps more importantly, there are many companies that are building
> TCP in silicon ASICs.  This should make TCP's performance comparable
> to Fibre Channel.  Both TCP/IP and FC provide about the same
> functionality ... reliable, in-order transmission.  
> 
> The bottom line is that FC is done in hardware while TCP has
> traditionally been done in software. Therefore, previous performance
> numbers are not going to be fair.  Once TCP is in silicon, its
> performance should be roughly equal to FC.
> 
>  >> c.  The maximum throughput of a GE TCP/IP connection is 
> 768 Mps, which =
>  >> is too slow to support storage I/O operations.
> 
>  I believe there are higher numbers (especially with Jumbo
> Frames). Alteon's web site show's 920 Mbps.  Microsoft and Duke
> University have both shown TCP performance o 1Gb+/s performance over
> other networks.
> 
>   BTW, why is 768 Mbps too slow for storage.  Many apps (e.g.,
> transaction workloads) are I/O's per second bound, not bandwidth
> bound.  Also, even if storage over IP/ether is a bit slower than FC,
> the benefits of leveraging IP's infrastructure (i.e., routers,
> switches, NICs, network management, networking people) is a huge
> advantage.  
> 
>  There is also the issue of SCSI over TCP/IP in the SAN vs. the
> LAN/WAN.  Some companies, focusing on the SAN, are building
> SCSI/lightweight transport/IP while others, focusing on the WAN,
> propose SCSI/TCP/IP.  It may be the case that SAN and WAN traffic use
> different transport protocols to gain a bit of extra performance in
> the SAN.  
> 
>  >> Is any of this true?
>  >> 
>  >> 2.  Adaptec has posited a replacement for TCP called STP 
> for use as a =
>  >> transport for storage.  Does anyone know anything about this?
> 
>     From Paul von Stamwitz's posting to the ips mailing list ...
>    
>       The link to the SEP draft is
>       http://www.ietf.org/internet-drafts/draft-wilson-sep-00.txt
>    
>       The press release is at:
> 	http://www.adaptec.com/adaptec/press/release000504.html
>    
>     The demo shows a Gb ethernet controller transporting SCSI 
> traffic to several
>     targets through an off-the-shelf 100TX switch with a Gb  
> uplink. The targets
>     are ethernet to U160 SCSI bridges with one or more SCSI  
> drives attached. The
>     host controller runs under NT4.0 at appears to the OS as 
> a  SCSI host bus
>     adapter.
>    
>     The architecture is based on Adaptec's SCSI Encapsulation Protocol
>     (SEP).  SEP is mapped on top of TCP/IP or a light-weight transport
>     protocol specifically designed for SANs.
>     
>     An SEP overview was presented at the IPS BOF in Adelaide 
> last  month and an
>     internet draft on SEP was submitted to IETF this week. I 
> will  forward the
>     link as soon as it becomes available. This draft is informational
>     only and intended to aid in this group's work toward an industry
>     standard SCSI transport protocol over IP networks.
> 
> 
>  >> 3.  Current discussions of the SCSI over IP protocol seem 
> to ignore the =
>  >> issue of TCP or any other transport protocol.  Does anyone know =
>  >> definitively what transport is being suggested by the 
> IBM/Cisco crowd?
> 
>    Current SCSI over IP discussions are not ignoring TCP ... they are
>    definitely considering TCP as the primary transport.  See the ips
>    web site at:
>  
     http://www.ece.cmu.edu/~ips

 >> 
 >> 4.  Another storage company is looking at Reliable UDP as a substitute =
 >> for TCP in storage data transfers.  Where can I learn more about this =
 >> protocol, which I am told was introduced many years ago by Cisco?

  Companies to look at include:

     nishansystems.com
     interprophet.com
     san.com
     arkresearch.com

  Also, I believe that the IETF IP over FC working group is now
looking at FC over IP.



dave...........

David Nagle
Director, Parallel Data Lab
Senior Reseach Computer Scientist
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-3898 (office)
412-268-3890 (fax)
http://www.ece.cmu.edu/~bassoon


------- End of Forwarded Message

Haagens, Randy (E-mail).vcf