Skip to main content

On the Feasibility of Simulation-Driven Portfolio Scheduling for Cyberinfrastructure Runtime Systems...

by Henri Casanova, Yick Wong, Loic Pottier, Rafael Ferreira Da Silva
Publication Type
Conference Paper
Book Title
Job Scheduling Strategies for Parallel Processing. JSSPP 2022. Lecture Notes in Computer Science
Publication Date
Page Numbers
3 to 24
Publisher Location
Cham, Switzerland
Conference Name
25th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2022)
Conference Location
Lyon, France
Conference Sponsor
Conference Date

Runtime systems that automate the execution of applications on distributed cyberinfrastructures need to make scheduling decisions. Researchers have proposed many scheduling algorithms, but most of them are designed based on analytical models and assumptions that may not hold in practice. The literature is thus rife with algorithms that have been evaluated only within the scope of their underlying assumptions but whose practical effectiveness is unclear. It is thus difficult for developers to decide which algorithm to implement in their runtime systems.

To obviate the above difficulty, we propose an approach by which the runtime system executes, throughout application execution, simulations of this very execution. Each simulation is for a different algorithm in a scheduling algorithm portfolio, and the best algorithm is selected based on simulation results. The main objective of this work is to evaluate the feasibility and potential merit of this portfolio scheduling approach, even in the presence of simulation inaccuracy, when compared to the traditional one-algorithm approach. We perform this evaluation via a case study in the context of scientific workflows. Our main finding is that portfolio scheduling can outperform the best one-algorithm approach even in the presence of relatively large simulation inaccuracies.