Deep Learning Techniques Improvement for Time Series Forecasting

Abstract

Time-series forecasting (TSF) is crucial for optimising decision-making across various domains, including energy systems, finance, healthcare, and engineering. However, traditional forecasting models often struggle with the complexities of Time-Series Data (TSD), such as nonlinear dependencies, spatial-temporal correlations, and evolving patterns in dynamic environments. This thesis presents a comprehensive Deep Learning (DL) based forecasting framework that integrates ensemble learning, guided networks, interactive convolution, self-supervised learning, transformer models, meta-learning strategies, and Bayesian Optimisation (BO) to enhance predictive accuracy, adaptability, and computational efficiency. To improve forecasting stability and generalisation, this research develops an EL framework that optimally combines multiple DL architectures. To further enhance the ensemble predictions, a guided network is introduced, leveraging an unsupervised learning strategy to dynamically adjust model outputs and mitigate overfitting. This guided network ensures that the ensemble framework effectively adapts to diverse time-series patterns by optimising the weighting of individual models. A guided ensemble network, refer as GATE, shows significant gains, with up to 17.5% improvement over Recurrent Neural Network (RNN) and 20.9% over Long Short-Term Memory (LSTM) in some cases. These findings underscore the effectiveness of GATE in enhancing TSF performance, particularly in reducing forecasting errors and improving overall model reliability. Traditional forecasting models often struggle to represent the intricate spatial-temporal dependencies and nonlinear characteristics inherent in renewable energy generation. To address these challenges, we propose two advanced DL frameworks: LSTM-SPAIC for solar power forecasting (SPF) and ASPEN-WIND for wind power forecasting (WPF). LSTM-SPAIC incorporates Interactive Convolution Blocks (ICB) to enhance spatial feature extraction, LSTM networks to capture extended temporal dependencies, and attention mechanisms to dynamically prioritize essential features. Additionally, adaptive spectral analysis is employed to refine temporal pattern recognition and reduce noise, further improving predictive accuracy. Empirical analysis demonstrates that LSTM-SPAIC improves forecasting precision by reducing Root Mean Square Error (RMSE) by 5–12% and Mean Absolute Error (MAE) by 5–8% on average, outperforming conventional DL approaches. ASPEN-WIND, designed for WPF, integrates an Adaptive Spectral Block (ASB) to extract multi-scale temporal patterns and filter out noise, while ICB modules enable robust spatial-temporal feature learning. LSTM networks are utilized to capture long-term dependencies, and self-supervised learning is introduced to enhance the model’s ability to generalize with limited labelled data. Performance evaluations on real-world wind farm datasets indicate that ASPEN-WIND achieves substantial improvements in forecasting accuracy, reducing RMSE by 6–14% and MAE by 7–10% on average, surpassing both conventional and state-of-the-art DL models across various forecasting horizons. Recurrent-based networks, such as LSTM and Gated Recurrent Unit (GRU), often struggle to process long sequences of time-series data due to their inherent limitations in capturing long-range dependencies and handling vanishing gradient issues. In contrast, transformer-based architectures offer a more effective solution by leveraging self-attention mechanisms to model complex temporal patterns over extended forecasting horizons. A Transformer-based forecasting framework for both WPF and short-term load forecasting is introduced, with BO integrated to enhance predictive accuracy and computational efficiency. For WPF, the proposed Enhanced WPF Transformer model effectively models complex wind power fluctuations, demonstrating performance improvements ranging from 5\% to 20\% over RNN-based models. These improvements highlight the model's ability to capture long-range dependencies and spatial-temporal correlations, making it a robust alternative for WPF. Similarly, for short-term load forecasting (LF), the Transformer-BO model is introduced, leveraging BO to explore hyperparameter configurations and adapt to rapid fluctuations in electricity demand. The Transformer-BO model is evaluated against Convolution Neural Network (CNN)-BO and LSTM-BO using historical load data and exogenous variables, demonstrating forecast accuracy improvements between 2% and 33%, with the most significant enhancements observed across multiple forecasting horizons. A reliable long-term time-series forecaster is highly required in practice but comes with many challenges, such as low computational and memory footprints as well as robustness against dynamic learning environments. We propose Meta-Transformer Networks (MANTRA) to deal with the dynamic long-term TSF tasks. MANTRA relies on the concept of fast and slow learners, where a collection of fast learners learn different aspects of data distribution while adapting quickly to changes. A slow learner tailors suitable representations to fast learners. Fast adaptations to dynamic environments are achieved using the Universal Representation Transformer (URT) layers, producing task-adapted representations with a small number of parameters. Our experiments using four datasets with different prediction lengths demonstrate the advantage of our approach, with at least 3% improvements over the baseline algorithms for both multivariate and univariate settings. The proposed methodologies are rigorously evaluated on multiple real-world datasets, including renewable energy forecasting (solar and wind power) and other time-series applications. Experimental results demonstrate that the developed models consistently outperform state-of-the-art DL methods, achieving substantial improvements in predictive accuracy, robustness, adaptability, and interpretability. The findings of this research contribute to the advancement of TSF by offering scalable, resilient, and high-performing DL solutions. These contributions provide valuable insights for industries reliant on precise forecasting, supporting data-driven decision-making in complex, dynamic, and resource-constrained environments.