May 2023

Conference Paper

Parallel I/O Evaluation Techniques and Emerging HPC Workloads: A Perspective

By:
Neuwirth, Sarah; Paul, Arnab Kumar
Page Number:
671-679
Book Title:
2021 IEEE International Conference on Cluster Computing (CLUSTER)
Publication Date:
May 2023
Publisher Location:
IEEE, New Jersey, United States of America
Conference Name:
IEEE CLUSTER: Workshop on Re-envisioning Extreme-Scale I/O for Emerging Hybrid HPC Workloads
Conference Location:
Portland, Oregon, United States of America
Conference Sponsor:
IEEE
View DOI Listing:
https://doi.org/10.1109/Cluster48925.2021.00100

Abstract

Emerging workloads such as artificial intelligence, big data analytics and complex multi-step workflows alongside future exascale applications are anticipated future HPC workloads, which will result in a more diverse I/O system workload and even less predictable I/O behavior and access patterns. Along with the ever increasing gap between the compute and storage performance capabilities, the in-depth understanding of extreme-scale I/O behavior and the I/O performance modeling and prediction are essential tools of the large-scale I/O evaluation process for addressing the needs of extreme-scale hybrid workloads. In this survey article, we focus on the state-of-the-art of the I/O behavior and performance analysis process for HPC systems in a 5-year time window and identify future research challenges.