Multivariate Time-Series Missing Data Imputation with Convolutional Transformer Model

Abstract

The rapid progress in artificial intelligence technologies has significantly impacted the global economy, driving transformative changes in manufacturing and giving rise to intelligent manufacturing. In this context, multivariable time-series data have become an essential resource for modern industries. This paper introduces the Point Energy Technology, an advanced system for energy monitoring and data acquisition developed by our team. The system has been successfully deployed with several industrial partners, including a combined heat and power system in a local industrial park. Despite its capabilities, data loss remains a persistent issue, which is often caused by measurement or transmission errors during the data collection and transfer stages. These errors result in the loss of vital data samples for effective process monitoring and control. To tackle this issue, we present a convolutional transformer imputation model that is based on self-attention to generate missing data samples. This model effectively captures both historical and future sequence information through an enhanced masking mechanism while also incorporating local dependency information through the symmetrically balanced use of convolution and self-attention. To evaluate the performance of the proposed model against classical models, the energy-related data from a local industrial park were used in this experiment. Considering the real-world conditions, the missing data were categorized into two types: continuous missing and random missing. The experimental results demonstrate that our model produced high-quality data samples, effectively compensating for gaps in the multivariable time-series data.