Performance Characterization of Containerized HPC Workloads

Abstract

In this paper, we address the complex challenges that arise within existing high performance computing (HPC) frameworks as we approach the exascale era. On one side, highly optimized, heterogeneous hardware systems coexist with HPC-unexperienced scientists with increasing demand for compute and data capacity. We propose containerization as a key concept to shift the focus back to the actual domain science, enabling an efficient usage of the compute systems and removing incompatible dependencies, unsupported subprograms or compilation challenges. To this end, we provide a methodology to determine, analyze and evaluate characteristic parameters of containerized HPC applications to fingerprint the overall performance of arbitrary containerized applications. The methodology comprises the performance parameter definition and selection, a measurement method to minimize overhead, and a fingerprinting algorithm to enable characteristics comparison and mapping between application and target system. We apply the methodology to benchmark applications to demonstrate its capability.