Transformer and long short-term memory networks for long sequence time sequence forecasting problem

Abstract

The long sequence time-sequence forecasting problem attracts a lot of organizations. Many prediction application scenes are about long sequence time-sequence forecasting problems. Under such circumstances, many researchers have tried to solve these problems by employing some models that have proved efficient in the Natural Language Processing field, like long short term memory networks and Transformers, etc. And there are a lot of improvements based on the primary recurrent neural network, and Transformer. Recently, a model called informer which is made for the LSTF was proposed. This model claimed that it improves prediction performance on the long sequence time-series forecasting problem. But in the later experiments, more and more researchers found that informers still cannot handle all the long sequence time-sequence forecasting problems. This paper is going to look at how datasets effect the performance of different models. The experiment is carried out on the Bitcoin dataset with four features and one output. The result shows that the Informer (transformer-like model) cannot always perform well so that sometimes choosing models with simple architecture may gain better results.