Measurement and Analysis of TCP Throughput Collapse in Cluster-based Storage Systems

6th USENIX Conference on File and Storage Technologies (FAST '08). Feb. 26-29, 2008. San Jose, CA. Supercedes Carnegie Mellon University Parallel Data Lab Technical Report CMU-PDL-07-105, September 2007.

Amar Phanishayee, Elie Krevat, Vijay Vasudevan, David G. Andersen, Gregory R. Ganger,
Garth A. Gibson, Srinivasan Seshan

School of Computer Science
Dept. Electrical and Computer Engineering
Carnegie Mellon University
Pittsburgh, PA 15213

Cluster-based and iSCSI-based storage systems rely on standard TCP/IP-over-Ethernet for client access to data. Unfortunately, when data is striped over multiple networked storage nodes, a client can experience a TCP throughput collapse that results in much lower read bandwidth than should be provided by the available network links. Conceptually, this problem arises because the client simultaneously reads fragments of a data block from multiple sources that together send enough data to overload the switch buffers on the client’s link. This paper analyzes this Incast problem, explores its sensitivity to various system parameters, and examines the effectiveness of alternative TCP- and Ethernet-level strategies in mitigating the TCP throughput collapse.

KEYWORDS: Cluster-based storage systems, TCP, performance measurement and analysis

FULL TR: pdf




© 2017. Last updated 15 March, 2012