Performance Evaluation 174 (Dec. 2026) 102591.
Zhouzi Li, Mor Harchol-Balter, Benjamin Berg*
Carnegie Mellon University
Pittsburgh, PA 15213
* UNC Chapel Hill
Modern data centers and cloud computing clusters are increasingly running workloads composed of malleable jobs. A malleable job can be parallelized across any number of cores, yet the job typically exhibits diminishing marginal returns for each additional core on which it runs. This can be seen in the concavity of a job’s speedup function, which describes the job’s processing speed as a function of the number of cores on which it runs. Examples of malleable jobs include machine learning tasks, parallelized across multiple GPUs, and large-scale database queries, parallelized across multiple servers.
Given the prevalence of malleable jobs, theoretical works have posed the problem of how to allocate a fixed number of cores across a stream of arriving malleable jobs so as to minimize the mean response time across jobs. We refer to this as the Core Allocation to Malleable jobs (CAM) problem. Although quite a few recent papers have been written on CAM, the problem remains challenging to solve optimally. In particular, prior theoretical work has only solved the CAM problem in the case where all jobs follow the same speedup function, with very little progress on multiple speedup functions.
This paper takes a new approach to CAM, looking at the problem in the mean field asymptotic regime. In this regime, we are able to optimally solve a much more general instance of CAM. We allow for any number of job classes, each with an arbitrary concave speedup function and weight. Furthermore, unlike prior work, we allow for general independent inter-arrival times and phase-type job sizes.
We derive two distinct mean field optimal policies, FW-CAM and GLAM. FW-CAM demonstrates a new intuition: in the mean field regime, individual job sizes are not relevant in finding an optimal policy. GLAM (Gittins-Like Allocation for Malleable jobs) is asymptotically optimal under certain common assumptions, and also serves as a good heuristic outside of the asymptotic regime by adapting dynamically to the current number of jobs in the system. Notably, none of the policies previously proposed in the literature are mean field optimal when jobs follow different speedup functions.
FULL PAPER: pdf