OpenStack cloud tuning for high performance computing

Abstract

High-Performance computing (HPC) is scarcely attempted in clouds because of slow and inefficient Inter-VM communication on the same server as well as huge latency between remote units. This was changed by introduction of ivshmem, a PCI device-based shared memory between VMs on the same server, but unfortunately, this mechanism became broken with Linux update few years ago. We have restored this shared memory system and made, for the first time, full cloud integration using latest versions of OpenStack, Linux, QEMU, libvirt and MPICH. Also, the analyses of different factors influencing both TCP/IP and ivshmem communication is presented along with tuning techniques that could significantly increase performance. Finally, we have created ivshmem communication channel that can replace standard Neutron TCP/IP network, resulting three to six times performance improvement.