Abstract
Traditional petascale applications, such as QMCPack, can scale their
computations to completely utilize modern supercomputers like Titan, but they
cannot scale their I/O. To preserve scalability, scientists cannot save data at
the granularity needed to enable scientific discovery and are forced to use
large intervals between two checkpoint calls. In this paper, we work to
increase the granularity of the I/O in QMCPack simulations without increasing
the I/O associated overhead or compromising the scalability of the simulations.
Our solution redesigns the I/O algorithms used by QMCPack to gather
finer-grained data at high frequencies and integrate the ADIOS API to select
effective I/O methods without major code changes. The extension of a tool such
as Skel to mimic the variable I/O in QMCPack allows us to predict the I/O
performance of the code when using ADIOS methods at the petascale. We show how
I/O libraries like ADIOS allow us to increase the amount of scientific data
extracted from QMCPack simulations at the granularity desired by the scientists
while keeping the I/O overhead below 10%. We also show how the impact of
checkpoint I/O for the QMCPack code using ADIOS is below 5% when using
preventive tactics for checkpointing at the petascale and beyond.
