PDL Abstract

A (In)Cast of Thousands: Scaling Datacenter TCP to Kiloservers and Gigabits

Carnegie Mellon University Parallel Data Lab Technical Report CMU-PDL-09-101, Feb 2009.

Vijay Vasudevan, Amar Phanishayee, Hiral Shah, Elie Krevat, David G. Andersen,
Gregory R. Ganger, Garth A. Gibson

School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213

http://www.pdl.cmu.edu/

This paper presents a practical solution to the problem of high-fan-in, high-bandwidth synchronized TCP workloads in datacenter Ethernets—the Incast problem. In these networks, receivers often experience a drastic reduction in throughput when simultaneously requesting data from many servers using TCP. Inbound data overfills small switch buffers, leading to TCP timeouts lasting hundreds of milliseconds. For many datacenter workloads that have a synchronization requirement (e.g., filesystem reads and parallel dataintensive queries), incast can reduce throughput by up to 90%.

Our solution for incast uses high-resolution timers in TCP to allow for microsecond-granularity timeouts. We show that this technique is effective in avoiding incast using simulation and real-world experiments. Last, we show that eliminating the minimum retransmission timeout bound is safe for all environments, including the wide-area.

KEYWORDS:Cluster-based storage systems, TCP, performance measurement and analysis

FULL TR: pdf