Docker Swarm Orchestration for High Performance Computing Cluster as Container

Abstract

This paper is aimed to investigate the inner-working of a high-performance computer (HPC) cluster running as containers on the docker platform. Docker Swarm as an orchestration tool is utilized to evaluate the scalability and the issues encountered during testbed deployment. Docker Swarm is a widely adopted open-source tool that enables the creation and management of a swarm of Docker nodes and the deployment and scaling of containerized applications across the swarm. HPC clusters, on the other hand, are specialized supercomputers that are designed to handle large and complex computational workloads. The HPC cluster for the example, is based on Mosix single system image (SSI) NUMA shared memory running in containers. The integration of Docker Swarm into HPC clusters would enable the orchestration of containerized workloads, which would bring benefits such as easier deployment, management of complex applications, and enhanced scalability. The technical issues encountered and the performance examples under different workloads and scalability scenarios are analyzed by comparing to the more traditional virtual machine (VM).