STFormer: A dual-stage transformer model utilizing spatio-temporal graph embedding for multivariate time series forecasting

Abstract

Multivariate Time Series (MTS) forecasting has gained significant importance in diverse domains. Although Recurrent Neural Network (RNN)-based approaches have made notable advancements in MTS forecasting, they do not effectively tackle the challenges posed by noise and unordered data. Drawing inspiration from advancing the Transformer model, we introduce a transformer-based method called STFormer to address this predicament. The STFormer utilizes a two-stage Transformer to capture spatio-temporal relationships and tackle the issue of noise. Furthermore, the MTS incorporates adaptive spatio-temporal graph structures to tackle the issue of unordered data specifically. The Transformer incorporates graph embedding to combine spatial position information with long-term temporal connections. Experimental results based on typical finance and environment datasets demonstrate that STFormer surpasses alternative baseline forecasting models and achieves state-of-the-art results for single-step horizon and multistep horizon forecasting.