Towards Cost-Efficient Edge Intelligent Computing With Elastic Deployment of Container-Based Microservices

Abstract

With the tremendous growth of the Internet of Things (IoT), big data, and artificial intelligence (AI), the edge computing-based service paradigm has been introduced to meet the increasing demand of applications. To provide efficient computing services at the network edge, the algorithms and applications are generally deployed based on the container-based microservice strategy, which significantly impacts the system efficiency and QoS. Considering the fundamental system uncertainties, including the dynamic workload and service rate, we investigate how to minimize the long-term system cost through the elastic microservice deployment in this paper. To this end, we formulate the container-based microservice deployment as a stochastic optimization problem to minimize the system cost while maintaining the system QoS and stability. We develop a cost-aware elastic microservice deployment algorithm to solve the formulated problem, which balances the tradeoff between system cost and QoS. Our algorithm makes the real-time decisions based on current queue backlogs and system states without predicting the future knowledge. Finally, we conduct the theoretical analysis and extensive simulations based on data traces from the ResNet-50 model-based visual recognition application. The results demonstrate that our algorithm outperforms the baseline strategies with respect to the system cost, queue backlogs, and the number of Pod replicas.