Special Instruction Set Processor for Convolutional Neural Network Based on RISC-V

Abstract

The x86-based and ARM-based CPU are limited by the patent authorization,which increases their customization cost and reduces the flexibility.To address the problem,this paper chooses the open-source instruction set architecture,RISC-V,to build an special instruction set processor for Convolutional Neural Network(CNN) used in the Internet of Things(IoT).The processor uses the custom extended instructions to call the accelerator to speed up the convolution and pooling operations of lightweight CNN,improving the power efficiency of terminal devices.In this process,the information of each layer of CNN is configured to control the accelerator to perform grouping operations,so as to adapt to the input data of different sizes.At the same time,the data path of the accelerator is adjusted,and the time-consuming operations are operated separately or in combination to adapt to different lightweight networks.The verification results on the FPGA platform show that this processor delivers a power consumption of 1.966 W when inferring SqueezeNet at 100 MHz.The inference takes about 40.89 ms,which is less than the single-core mobile phone processors take.Also,it reduces the consumption of resources and power,demonstrating an obvious advantage in performance power ratio compared with AMD Ryzen7 3700X,NVIDIA RTX2070 Super and Qualcomm Snapdragon 835.