ALCC 2015 Final Report for Hobbes Operating System and Runtime Research for Extreme Scale

Abstract

The Hobbes project is a collaboration of four national laboratories and eight universities that has been selected for funding by the Exascale Operating and Runtime Systems program under DOE’s Once of Advanced Scientific Computing Research (ASCR). The goal of Hobbes is to provide a system software environment that enables application composition through lightweight virtualization. The focus on application composition is based on the realization that continuing to manage an extreme-scale systems through space sharing and batch scheduling is becoming and will soon be insufficient. Application workflows are evolving to be much more complex and will require more suitable usage models. Ensemble calculations for uncertainty quantification, large graph analytics, multi-materials and multi-physics applications are just a few examples that are driving the need for new system software interfaces and mechanisms for enabling more sophisticated workflows and sharing of memory and computational resources. Rather than providing a single unified OS/R that supports several parallel programming models, Hobbes is leveraging lightweight virtualization that provides the flexibility to construct and execute custom OS/R environments. While much of the exploration and development of the Hobbes software environment can be done at small scale, evaluating the scalability of OS/R interfaces and mechanisms at large scale is crucial. Indeed, much of the focus of extreme-scale system software in the last decade has been on measuring and characterizing the impact the OS can have on application scalability and evaluating approaches that eliminate or mitigate this impact. Given the expanded role the OS/R must undertake, periodic evaluation of implementation techniques at scale will be critical to meeting the challenges facing extreme-scale systems in the next five years. Through the competitive ALCC Program, Hobbes was granted a 40.0 Million core-hour award to conduct leading edge research.