NEURAL NETWORK ARCHITECTURE BASED ON GRAPH CODES

Abstract

One of the important achievements of the theory of error-correcting coding is the discoveryof graph codes and their important subset - low-density parity check codes (LDPC codes). Usingthe parity check matrix of the code on the graph, one can obtain a Markov random field. LDPCcode can be embedded in an Ising model (a type of Markov random field) by using a torus topologywith negative curvature. In this case, codewords correspond to saddle points (extrema) in themodel, and trappin sets correspond to local minima. The use of LDPC codes with an increasedcode distance allows for maximum separation of saddle points, and thus increases the noise resistanceof the neural network and the representation power. At the same time, the block andsparse structure, characteristic of a torus of negative curvature, simplifies multiplexing and reducesthe number of trainable parameters of the neural network. The aim of the research is toreduce the computational complexity and increase the accuracy of neural networks through theuse of a priori structural (quasi-cyclic) sparse graphs for a wide class of machine learning problemson Markov random fields. The paper presents a new approach that allows the synthesis ofneural network architectures based on graph codes. The proposed approach provides an effectiverepresentation of Markov random fields through the use of QC-LDPC matrices and tensors.The proposed approach allows us to reduce the number of trainable parameters and logarithmicallyreduce the complexity of tensor multiplexing. The proposed approach provided an accuracyof 94.95% (1.72% to first place) of the binary classification problem “Pathfinder” of the “LongRange Arena” competition, with more than 5 times fewer parameters (multiplications). Applicationof the proposed approach to factorization problems on dense graphs, network problems, surfacemeshes, covariance matrices made it possible to increase the accuracy of reconstruction usingthe Frobenius metric in individual problems by more than 8 orders of magnitude in combinationwith simplifying the structure of the multiplexer.