Date: September 10, 1998
Time: Noon
Place: Wean Hall 8220
Speaker: Phil Koopman, Carnegie Mellon University
Measuring Operating System Robustness (early results from the CMU Ballista project)
Abstract:
Robustness is becoming more important as critical software increasingly affects our daily lives. Success in building robust software requires understanding and improving the robustness of the operating system API, but to date there has been no accurate, reproducible way to measure robustness. This talk presents the first full-scale, quantitative measurements of operating system robustness. Each of 15 different operating system's robustness is measured by automatically testing up to 233 POSIX functions and system calls with exceptional parameter values. The work identifies repeatable ways to crash operating systems with a single call, ways to cause task hangs within OS code, ways to cause task core dumps within OS code, failures to implement defined POSIX functionality for unusual conditions, and false indications of successful completion in response to exceptional input parameter values. Overall, only 55% to 76% of tests performed were handled robustly, depending on the operating system being tested.
This work is part of the Ballista project, which seeks to develop ways to automatically test for and improve the robustness of general-purpose software modules. This Fall we will be beta testing a web server that remotely tests the robustness of software modules beyond just operating system calls. More information and an on-line OS testing demo are available at http://www.ices.cmu.edu/ballista
SDI / LCS Seminar Questions?
Karen Lindenfelser, 86716, or visit www.pdl.cmu.edu/SDI/