Skip to main content

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

Publication Type
Conference Paper
Book Title
2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Publication Date
Page Numbers
290 to 300
Publisher Location
New Jersey, United States of America
Conference Name
2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Conference Location
St. Petersburg, Florida, United States of America
Conference Sponsor
Conference Date

We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.