Sparse Structure Enabled Grid Spectral Mixture Kernel for Temporal Gaussian Process Regression

Abstract

We propose a modified spectral mixture (SM) kernel that serves as a universal stationary kernel for temporal Gaussian process regression (GPR). The kernel is named grid spectral mixture (GSM) kernel as we fix the frequency and variance parameters in the original SM kernel to a set of pre-selected grid points. The hyper-parameters are the non-negative weights of all sub-kernel functions and the resulting optimization task falls under the difference-of-convex programming. Due to the nice structure of the optimization problem, the hyper-parameters are solved by an efficient majorization-minimization method instead of the gradient descent methods. It turns out that the solution is sparse, which provides us with a principled guideline to identify the important frequency components of the data. Experimental results based on various classic time series data sets corroborate that the proposed GPR with GSM kernel significantly outperforms the GPR with SM kernel in terms of both the mean-squared-error (MSE) and the stability of the optimization algorithm.