Henggang Cui, Gregory R. Ganger, Phillip B. Gibbons
Carnegie Mellon University
MLtuner automatically tunes settings for training tunables— such as the learning rate, the momentum, the mini-batch size, and the data staleness bound—that have a significant impact on large-scale machine learning (ML) performance. Traditionally, these tunables are set manually, which is unsurprisingly error prone and difficult to do without extensive domain knowledge. MLtuner uses efficient snapshotting, branching, and optimization-guided online trial-and-error to find good initial settings as well as to re-tune settings during execution. Experiments show that MLtuner can robustly find and re-tune tunable settings for a variety of ML applications, including image classification (for 3 models and 2 datasets), video classification, and matrix factorization. Compared to state-ofthe- art ML auto-tuning approaches, MLtuner is more robust for large problems and over an order of magnitude faster.
FULL PAPER: pdf