Research on Resource Scheduling Optimization Method of Hadoop Yarn

Abstract

In order to solve the Hadoop Yarn scheduling problem, improve the efficiency of cluster job, by considering the advantages of ant colony algorithm and simulated annealing algorithm; we proposed a Hadoop resource scheduling algorithm ACOSA. In ACOSA, we initialize the pheromone matrix of ACOSA by using the attribute information of load, memory, and CPU speed obtained through the heartbeat message transfer mechanism. After getting a group of optimal solution, the path was optimized, and the pheromone of solution was updated by the simulated annealing algorithm. Finally, the simulation experiment on CloudSim platform shows that the efficiency of job execution is improved by adopting ACOSA algorithm for resource scheduling.