Designing RISC-V Instruction Set Extensions for Artificial Neural Networks: An LLVM Compiler-Driven Perspective

Abstract

The demand for Artificial Intelligence (AI) based solutions is exponentially increasing in all application fields, including low-power devices on the edge. However, due to their limited computational capabilities, these devices, which run Central Processing Units (CPUs) tailored to embedded applications, are typically not optimized to run complex neural networks. Providing ad-hoc extensions to the instruction set architecture of a RISC-V processor can be a viable solution to address this issue. In this work, we propose the use of the PyTorch Graph Lowering (Glow)–LLVM toolchain to understand the impact of compiled code of AI models on a RISC-V machine and extend its instruction set to improve runtime performance. This approach allows code profiling, detection of computational bottlenecks, and provisioning of the necessary CPU enhancements to be implemented in the LLVM backend before hardware implementation. After profiling known Artificial Neural Networks (ANNs) quantized to int8 (particularly, a single perceptron, RESNET18, VGG11, and LENET5), we have identified and devised three additional instructions, we named LWM, LWA and LWS respectively indicating Load Word-and-Multiply, -Add, and -Subtract. As a result, we obtained an edge AI-oriented, significantly improved processor description in terms of inference time and program density, ready to be hardware-designed. For $128\times 128$ RGB images, the custom extensions enabled up to $13\times $ speed up compared to RV32I and $5\times $ compared to RV32IM, with a maximum of 11.7% lower code. Together with these findings the paper systematically highlights the main methodological steps to include new instructions in an LLVM backend.