Similarity based Deep Neural Networks

Abstract

Deep neural networks (DNNs) have recently attracted attention in various areas.Their hierarchical architecture is used to model complex nonlinear relationships in high-dimensional data.DNNs generally require large numbers of data to train millions of parameters.However, the training of a DNN with a small number of high-dimensional data can result in an overfitting.To alleviate this problem, we propose a similarity-based DNN that can effectively reduce the dimensionality of the data.The proposed method utilizes a kernel function to calculate pairwise similarities of observations as input, and the nonlinearity based on the similarities is then explored using a DNN.Experiment results show that the proposed method performs effectively regardless of the dataset used, implying that it can be applied as an alternative when learning a small number of high-dimensional data.