Robust Twin Extreme Learning Machine Based on Soft Truncated Capped L1-Norm Loss Function

Abstract

Currently, most researchers propose robust algorithms from different perspectives for overcoming the impact of outliers on a model, such as introducing loss functions. However, some loss functions often fail to achieve satisfactory results when the outliers are large. Therefore, the capped loss has become a better choice for researchers. The majority of researchers directly set an upper bound on the loss function, which reduces the impact of large outliers, but also introduces non-differentiable regions. To avoid this shortcoming, we propose a robust twin extreme learning machine based on a soft-capped L1-normal loss function (SCTELM). It uses a soft capped L1-norm loss function. This not only overcomes the shortcomings of the hard capped loss function, but also improves the robustness of the model. Simultaneously, to improve the learning efficiency of the model, the stochastic variance-reduced gradient (SVRG) optimization algorithm is used. Experimental results on several datasets show that the proposed algorithm can compete with state-of-the-art algorithms in terms of robustness.