Constructive Multivariate Approximation with Sigmoidal Functions and Applications to Neural Networks

Abstract

In this paper, we show how to use sigmoidal functions in order to generate approximation operators for multivariate functions of bounded variation. We start with Lebesgue-Stieltjes type convolution operators, then — via numerical quadrature — we pass over to point-evaluation operators and give local and global approximation results for them. In the following, we discuss an important application of our results to neural networks with one hidden layer consisting of so-called sigma-pi units. In this context, our results should be seen as an explicit constructive contribution to Kolmogorov's well-known mapping existence theorem for three-layer feedforward neural networks. At the end, we apply our operators, resp. networks, to a special test function in order to get some concrete idea of their behaviour.