In-memory Computing Architectures for Energy-efficient AI

Abstract

The exponential growth of AIespecially deep learning and generative AIis severely constrained by the "memory wall" in von Neumann architectures, where frequent data movement between processors and memory consumes up to 90% of energy and creates critical latency bottlenecks. To address these limitations, this paper examines in-memory computing (IMC) as a transformative paradigm that co-locates computation and storage, targeting energy-efficient acceleration for AI workloads from edge inference to large-scale training. The analysis of DRAM, SRAM, and non-volatile memory (NVM) approaches reveals significant breakthroughs: capacitorless IGZO DRAM enables monolithic 3D-stacked, multibit arrays; ReRAM/PCM crossbars deliver ultra-efficient analog multiply-accumulate operations; and heterogeneous architectures (e.g., integrated analog-digital tiles with 2D mesh interconnects) achieve 2264 TOPS/W efficiency40140 higher than GPUs. However, challenges persist in precision management, device variability, system programmability, and 3D integration scalability. This study concludes that IMC is pivotal for sustainable AI, potentially reducing operational carbon footprints by 10100 through eliminated data movement. By overcoming current limitations via hybrid designs and standardized interfaces, IMC can extend beyond neural networks to graph processing and scientific computing, establishing itself as the cornerstone of future intelligent systems from edge to cloud.