Skip to main content

Comparative evaluation of deep learning workloads for leadership-class systems...

Publication Type
Conference Paper
Journal Name
BenchCouncil Transactions on Benchmarks, Standards and Evaluations
Publication Date
Page Number
Conference Name
2021 BenchCouncil International Symposium on Benchmarking, Measuring and Optimizing
Conference Location
N/A, Tennessee, United States of America
Conference Sponsor
Conference Date

Deep learning (DL) workloads and their performance at scale are becoming important factors to consider as we design, develop and deploy next-generation high-performance computing systems. Since DL applications rely heavily on DL frameworks and underlying compute (CPU/GPU) stacks, it is essential to gain a holistic understanding from compute kernels, models, and frameworks of popular DL stacks, and to assess their impact on science-driven, mission-critical applications. At Oak Ridge Leadership Computing Facility (OLCF), we employ a set of micro and macro DL benchmarks established through the Collaboration of Oak Ridge, Argonne, and Livermore (CORAL) to evaluate the AI readiness of our next-generation supercomputers. In this paper, we present our early observations and performance benchmark comparisons between the Nvidia V100 based Summit system with its CUDA stack and an AMD MI100 based testbed system with its ROCm stack. We take a layered perspective on DL benchmarking and point to opportunities for future optimizations in the technologies that we consider.