【TSF 文献阅读 02】Dlinear | Are Transformers Effective for Time Series Forecasting?

摘要

Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task.

Despite the growing performance over the past few years, we question the validity of this line of research in this work.

Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence.

However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss.

最近，基于Transformer的解决方案在长期时间序列预测（LTSF）任务中出现了激增。尽管过去几年性能不断增长，我们在这项工作中质疑这一研究方向的有效性。具体来说，Transformer可以说是提取长序列中元素之间语义相关性最成功的解决方案。

然而，在时间序列建模中，我们需要提取有序连续点集中的时间关系。虽然使用位置编码和令牌将子序列嵌入到Transformer中有助于保留一些顺序信息，但置换不变的自注意力机制的本质不可避免地导致时间信息的丢失。