|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] IPS Issues/Requirements Document
Appended is a first draft of the IPS Issues/Requirements document.
It was submitted to the IETF as an Internet Draft on March 10, 2:42 PST
Unfortunately, it missed the cutoff time of 5:00 EST by 42 minutes, so
it was NOT processed.
All comments and suggestions for improvements all welcome by responding
to the IPS list, or to me directly.
The timeline for the final draft is March 20, so the more feedback I can
have this week, the better.
Dave Nagle,
If you feel that it is appropriate, please cross post it to other
storage groups (SNIA, NSIC, T10, etc.) to get them involved and post it
to the web site.
Thank you,
Paul von Stamwitz
-----------------------------
Internet Draft P. von Stamwitz
<draft-von-ipsissues-01.txt> D. Wilson
Expires 10 September 2000 Adaptec
C. Sapuntzakis
Cisco Systems
D. Lee
3Com
March 2000
IP Storage
Issues and Requirements
This document is an Internet-Draft and is NOT offered in accordance
with Section 10 of RFC2026, and the author does not provide the
IETF with any rights other than to publish as an Internet-Draft.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Status of this Memo
This document specifies issues and requirements for the IP Storage
BOF and any future working group. It is a first draft and has not
been reviewed by the Internet community. A subsequent revision is
likely after discussion and suggestions for improvements have been
submitted. The next revision is planned to be submitted by March
22, 2000. Please submit all comments to the ips@ece.cmu.edu or
paulv@corp.adaptec.com. Distribution of this memo is unlimited.
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 1]
IP Storage Issues and Requirements March 2000
Table of Contents
Abstract ............................................................. 2
1. Introduction and Motivation ....................................... 3
2. Requirements and issues for storage ............................... 3
2.1. Data Integrity and Reliability ................................. 3
2.2. Ability to span large distances ................................ 3
2.3. Performance .................................................... 4
2.3.1. Performance vs. Connectivity Solution ....................... 4
2.3.2 Competitive with existing architectures ...................... 4
2.3.3 CPU Utilization .............................................. 5
2.3.4 Storage Stack vs. Network Stack Processing ................... 5
2.3.5. Flow control at the Storage Protocol ........................ 5
2.4. Storage Management ............................................. 6
2.4.1. Sharing Storage vs. Data .................................... 6
2.5. Security ....................................................... 6
2.6. Device Discovery and Configuration ............................. 6
3. Optimum Criteria for Solutions .................................... 7
4. Candidates for Solutions .......................................... 8
4.1. Transport ...................................................... 8
4.1.1. TCP ......................................................... 8
4.1.2. UDP ......................................................... 8
4.1.3. Scheduled Transfer (ST) ..................................... 8
4.1.4. Other? ...................................................... 8
4.2 Storage Protocol ................................................ 8
4.2.1. SCSI ........................................................ 8
4.2.2. FCP ......................................................... 9
4.2.3. ATA ......................................................... 9
4.2.4. NFS ......................................................... 9
4.2.5. Other? ...................................................... 9
5. An example solution ............................................... 9
6. Conclusion ........................................................ 9
Abstract
Recently, there has been increasing interest in investigating ways on
how to deliver enhancements to existing networking technology for
storage solutions. To facilitate exploration of the issues, a Birds
of a Feather (BoF) session has been scheduled during the IETF
Meetings of March, 2000. The BoF session will be held on March 29,
2000 at 3:30PM in Adelaide, Australia.
The intent of the memo is to facilitate discussion and achieve
consensus on the best approach and the key issues that need to be
investigated before storage over IP becomes a reality.
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 2]
IP Storage Issues and Requirements March 2000
1. Introduction and Motivation
Recently, the trend toward merging networking and storage
interconnect technologies has accelerated. This merging has given
rise to new technologies such as Storage Area Networks (SANs) and
Network Attached Storage (NAS.) Other technologies, such as Object
Based Storage Device (OBSD) are being discussed. Due to the increased
usage of the Internet, storage demands will continue to increase. In
light of this, it is reasonable to assume that the merging of
networking and storage will continue.
While file storage protocols like NFS and AFS have long operated over
IP-based networks, block storage protocols such as SCSI and ATA have
yet to make the leap from non-IP based interconnects to IP-based
networks. With the emergence of gigabit and 10-gigabit IP networks,
transporting block storage over IP networks is increasingly
attractive. While block storage based SAN implementations have
applied networking concepts to storage technology, another
perspective is to apply storage principles and concepts to existing
networking technology.
The IP Storage architecture offers a framework within which existing
network infrastructures can be used as a high-quality storage
subsystem interconnect. The goal is to provide current and future
storage requirements while leveraging the broad installed base as
well as the inherent manageability, scalability, and availability of
the network.
2. Requirements and issues for storage
2.1. Data Integrity and Reliability
This is an absolute requirement of storage. Whatever solution is
used, it must be shown empirically that it is safe and reliable.
2.2. Ability to span large distances
Block storage has generally been relegated to the subnet. As storage
demands increase, the need for remote backup and data mirroring for
storage consolidation and disaster recovery is becoming increasingly
important. The ability to transfer storage over an existing network
infrastructure makes IP-based storage very attractive. Another
attractive feature of IP Storage is the ability to bridge subnets via
a high speed interconnect. While information between subnets are
shared today through file transfers over the communications network,
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 3]
IP Storage Issues and Requirements March 2000
the ability to directly access data resident in other subnets may be
useful in some applications.
The bridging of subnets raises the issue of supporting legacy
installations. For example, one application of IP Storage could be
the bridging of multiple FibreChannel-based SANs. The IP Storage
architecture should consider ways that easily adapt to legacy
installations. Adequate liaisons need to be in place with other
standards bodies such as ANSI T10, T11, and T13 (SCSI, FC, and ATA
respectively) to ensure interoperability.
In the future, it may be highly desirable for the subnets themselves
to be based on IP Storage. However, optimal solutions for the subnet
raise a different, though overlapping, set of issues and
requirements. Also, the IETF may not be the appropriate venue for
optimizing IP Storage for the subnet. In order to keep the issues
clear and relevant to the Internet community and the goals reasonably
specific, a decision must be made early on as to whether issues
related to LANs or SANs are to be addressed.
2.3. Performance
In the storage market, speed sells. Projected access times and
sequential access rates for disk drives, assuming historical annual
improvement rates, will approach 4 milliseconds and 100
megabytes/second respectively within the next 5 years. But,
performance requirements vary greatly depending on the application.
For applications which access data sequentially, or in very large
data blocks, low latencies and high bandwidth will be required. On
the other hand, transaction processing applications will still be
limited by disk access time to about 350 IOPS per drive, or one to
five megabytes a second depending on request size. Furthermore, the
caching of data at various points within the system can effectively
eliminate the physical limitations of rotating media, resulting in
memory to memory speeds. Before performance requirements can be
discussed, certain issues need to be addressed.
2.3.1. Performance vs. Connectivity Solution
There needs to be an agreement on which problem IP Storage is
solving. While performance is always important, connectivity
solutions tend to have less severe performance requirements while
having a greater emphasis on ease-of-use or solving data access
problems.
2.3.2 Competitive with existing architectures
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 4]
IP Storage Issues and Requirements March 2000
If IP Storage is to provide storage solutions that are currently
provided by other architectures, then IP Storage must be competitive
in a cost/performance basis. Since it is assumed that the use of off-
the-shelf networking switches, routers, and wiring will produce
substantial cost savings due to the large volumes associated with the
use of such equipment for networks, the performance requirements need
only to meet those of other architectures in typical applications.
2.3.3 CPU Utilization
One of the benefits of the IP Storage architecture is the ability to
scale servers and storage devices independently. If bandwidth is
exceeded, host bus adapters can be added. But, if the CPU is
exceeded, then additional servers are required. Traditionally, CPU
utilization of storage architectures has been significantly lower
than that of networking. Methods for the reduction of network stack
processing overhead at the storage clients and servers need to be
considered and possibly incorporated into the protocol.
2.3.4 Storage Stack vs. Network Stack Processing
In the server, the storage stack has generally been more efficient
than the network stack for various reasons. The transport protocol is
simpler than that of networking, thus more readily able to be
incorporated in hardware. The server acts as the initiator, and
rarely (if at all) receives unsolicited commands. Read memory is
locked and the host bus adapter is able to transfer data directly
into destination memory via a host-supplied scatter/gather list. This
direct memory access is simplified by the fact that data is received
from the target in order. For IP Storage, it is probably unreasonable
to require in order delivery. However, a comparison of the two stacks
might be useful in developing ways of increasing the efficiency of
the protocol processing.
2.3.5. Flow control at the Storage Protocol
Whereas the server, in the typical storage stack, has read memory
pre-allocated by the operating system, the storage client has a
finite amount of memory to store write data. This requires some level
of flow control to avoid overrunning memory resources. For example,
parallel SCSI uses disconnects and Fibrechannel uses transfer ready
as methods of flow control. The flow control in the transport
protocol could handle this situation if each command had a unique
session. However, if a session could support multiple concurrent
commands (i.e. tagged queuing), then the transport's flow control
would effectively pause data flow for all commands, even if there are
available resources for some of the commands. It needs to be
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 5]
IP Storage Issues and Requirements March 2000
determined whether some flow control is necessary at the storage
protocol layer. If so, a control mechanism such as transfer ready may
produce undesirable latencies, especially in long distance
configurations; and, if possible, a safe mechanism could be found to
reduce this latency.
2.4. Storage Management
The ability to consolidate storage allows for many heterogeneous
servers to share the same storage pool. This also presents new
problems of management of that storage pool. By leveraging the
inherent manageability of IP as well as the experiences of the
Fibrechannel community, virtual LANs can be established using IP-
based tools to segment and allocate storage at the port, LUN, and/or
partition level. SNMP MIBs and/or CIM object models for storage
devices may be necessary.
2.4.1. Sharing Storage vs. Data
One advantage that NFS has over block storage protocol is the ability
to share files among multiple diverse server environments. Currently,
sharing of data using block storage is only done in a clustered
environment. Ongoing discussions in the Storage Networking Industry
Association (SNIA) and the ANSI committees regarding object-based
storage should be followed and appropriate liaisons established.
2.5. Security
Assigning IP addresses to storage clients can be alarming to those
tasked with ensuring data is protected from unwanted access. Work is
being done at the storage protocol to increase the level of access
control. It must be noted that these methods for restricting access
may assume a protected environment. In other words, the protocol may
guard against accidental access, but may not be a sufficient
safeguard against malicious access. Leveraging IPsec will provide
greater security in more open environments.
2.6. Device Discovery and Configuration
Communication networks, which assume that all attached devices are
peers and that any given device will want to communicate with only a
small subset of other devices. On the other hand, storage networks
are hierarchical, with a few Initiators in communication with many
targets, and with targets in communication only with initiators, or
with other targets under the control of an initiator. This storage
viewpoint requires that initiators be able to locate all the targets,
and be able to communicate with many or all of them. A decision must
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 6]
IP Storage Issues and Requirements March 2000
be made on which method to use for discovery: peer-to-peer or master-
slave. Any discovery protocol should be scalable up to systems of
hundreds of initiators and thousands of targets.
A negotiation protocol also needs to be defined to allow for auto-
configuration of devices. Adequate liaisons need to be in place with
other working groups in the IETF that are relevant to these areas
(e.g. zeroconf, LDAP, Service Location Protocol).
2.7. Quality of Service
Investigation should be done on how QoS can enhance storage solutions
running over networks. A strong partnership with IPv6 will ensure
that storage needs will be considered.
2.8 Storage Subsystem vs. Device
Many of the issues discussed can be addressed by the subsystem
vendors, but may be prohibitively expensive and/or complex in the
highly competitive disk drive market. Assuming the IP Storage
architecture is successful, it is highly likely that it will be
adopted by the drive manufacturers. A decision needs to be made on
whether integration at the drive level needs to be considered at this
time, or at some later date (or by some other organization.)
3. Optimum Criteria for Solutions
The following are some criteria that, though not necessary, would
increase the likelihood of success and aid in the quick adoption of
the IP Storage architecture in the marketplace:
- Leverage existing technology wherever possible, thus increasing
the likelihood of interoperability.
- The solution requires little or no modification to existing
operating systems, ensuring early deployment.
- Existing software tools and applications are compatible, for both
networking and storage. This allows for a high degree of
application specific testing prior to deployment.
- Deployment of the solution requires little or no additional
training.
- The solution has appeal in multiple segments of the market.
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 7]
IP Storage Issues and Requirements March 2000
4. Candidates for Solutions
4.1. Transport
4.1.2. TCP
TCP is ubiquitous on the Internet. It has many years of development
and enhancements. There is a strong comfort level with IS
administrators. It has proven interoperability in a wide range of
configurations. TCP has a great deal of support among the networking
community.
TCP also has the reputation of being large, software intensive, and
overly complicated. There are those in the storage community that are
skeptical that TCP can provide the required level of performance. In
addition, the checksum may not be adequate for ensuring data
integrity.
A detailed performance study should be done. A key factor is the
network stack processing overhead. New development of hardware
accelerated TCP could be pivotal.
4.1.3. UDP
UDP is used heavily in the LAN environment, but will have to win over
skeptics that it can run safely over the Internet/WAN.
4.1.4. Scheduled Transfer (ST)
Developed in the HIPPI organization, it provides high bandwidth, low
latency, and low CPU utilization. A SCSI encapsulation protocol has
already been proposed in ANSI T10. Will have to win over skeptics
that it can run safely over the Internet/WAN.
4.1.5. Other?
Any other proposed transport protocol should be considered.
4.2 Storage Protocol
4.2.1. SCSI
SCSI has two meanings. Logical SCSI refers to the command protocol.
Most storage stacks are based on the SCSI Architectural Model (SAM).
It is this logical command protocol that would need to be
encapsulated and transported over IP.
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 8]
IP Storage Issues and Requirements March 2000
SCSI also is equated with devices incorporating the SCSI Parallel
Interface (SPI). These drives have the largest deployment in the
server market. Since it is a parallel interface, bridging would
usually be done at the storage subsystem. Its interlocked protocol is
dissimilar in behavior to networking, but the newly approved
Information Unit phase may make the conversion simpler.
4.2.2. FCP
This is Fibrechannel's version of encapsulated SCSI command protocol.
Bridging can be done at either the storage subsystem or router.
4.2.3. ATA
There is interest here due to the cost differential between ATA and
SCSI drives. A serial version of ATA is also being developed. The
storage protocol would still be based on the SCSI model. This
interface accounts for approximately 85% of all disk drives sold
world-wide.
4.2.4. NFS
NFS is the current solution for accessing storage over the network.
Its strengths are ease-of-use, file sharing, and security. The
general assumption is that a block storage protocol should outperform
NFS in some applications, but that assumption needs to be tested.
Also, some applications (e.g. tape library) could benefit from a
block storage protocol. In order for a block-based solution to be
accepted, clear benefits over NFS (e.g. performance, direct access)
must be proven. Even then, NFS would still be the solution of choice
for those requiring ease-of-use and file sharing.
4.2.5. Other?
There may be other mappings of SCSI over serial interconnects. A more
general approach could also be explored. A class-based (disk, tape,
enclosure, etc.) message could be transported to the storage client
where it was then translated to the native interface(SCSI, FCP, etc.)
5. An example solution
A proposed solution which transports SCSI over TCP is available as an
internet draft under the name <draft-satran-iSCSI-02.txt>
6. Conclusion
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 9]
IP Storage Issues and Requirements March 2000
The critical success factors are:
- Data integrity and reliability are empirically demonstrated.
- Issues of security are satisfactorily addressed.
- Any block storage protocol must show clear differentiation from
NFS.
Expires 10 September 2000
IP Storage Issues and Requirements March 2000
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 5]
Von Stamwitz, Wilson, Sapuntzakis, Lee [Page 1]
Home Last updated: Tue Sep 04 01:08:17 2001 6315 messages in chronological order |