2020

Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

Xu, Mingxing, Dai, Wenrui, Liu, Chunmiao et al.

Understand

Traffic forecasting has emerged as a core component of intelligent transportation systems.

However, timely accurate traffic forecasting, especially long-term forecasting, still remains an open challenge due to the highly nonlinear and dynamic spatial-temporal dependencies of traffic flows.
In this paper, we propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) that leverages dynamical directed spatial dependencies and long-range temporal dependencies to improve the accuracy of long-term traffic forecasting.
Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies with self-attention mechanism to capture realtime traffic conditions as well as the directionality of traffic flows.

Built on

Z. Wu, S. Pan, G. Long, J. Jiang, and C. Zhang, “Graph wavenet for deep spatial-temporal graph modeling,” in Proc. 28th Int. Joint Conf. Artif. Intell. , Macao, China, Aug. 2019, pp. 1907–1913
1913
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput. , vol. 9, no. 8, pp. 1735–1780, Nov. 1997
1997
Earlier work this paper cites.
S. Makridakis and M. Hibon, “ARMA models and the Box-Jenkins methodology,” J. Forecast. , vol. 16, no. 3, pp. 147–163, May 1997
1997
Earlier work this paper cites.
J. Atwood and D. Towsley, “Diffusion-convolutional neural networks,” in Adv. Neural Inf. Process. Syst. 29 , Barcelona, Spain, Dec. 2016, pp. 1993–2001
2001
Earlier work this paper cites.
S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber, “Gradient flow in recurrent nets: The difficulty of learning long-term dependencies,” in A Field Guide to Dynamical Recurrent Neural Networks . IEEE Press, 2001, pp. 237–244
2001
Earlier work this paper cites.
C.-H. Wu, J.-M. Ho, and D.-T. Lee, “Travel-time prediction with support vector regression,” vol. 5, no. 4, pp. 276–281, Dec. 2004
2004
Earlier work this paper cites.
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” vol. 20, no. 1, pp. 61–80, Jan. 2008
2008
Earlier work this paper cites.
W. Liu, Y. Zheng, S. Chawla, J. Yuan, and X. Xing, “Discovering spatio-temporal causal interactions in traffic data streams,” in Proc. 17th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min. , San Diego, CA, USA, Aug. 2011, pp. 1010–1018
2011
Earlier work this paper cites.
W. Min and L. Wynter, “Real-time road traffic prediction with spatio-temporal correlations,” Transp. Res. Part C Emerg. Technol. , vol. 19, no. 4, pp. 606–616, Aug. 2011
2011
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Adv. Neural Inf. Process. Syst. 25 , Lake Tahoe, NV, USA, Dec. 2012, pp. 1097–1105
2012
Earlier work this paper cites.
M. Lippi, M. Bertini, and P. Frasconi, “Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning,” vol. 14, no. 2, pp. 871–882, Jun. 2013
2013
Earlier work this paper cites.

Similar

J. Wang, W. Deng, and Y. Guo, “New bayesian combination method for short-term traffic flow forecasting,” Transp. Res. Part C Emerg. Technol. , vol. 43, pp. 79–94, Jun. 2014
2014
Cited alongside, same era.
J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” in NIPS 2014 Workshop Deep Learn. , Montreal, QC, Canada, Dec. 2014
2014
Cited alongside, same era.
I. Sutskever, O. Vinyals, and V. L. Quoc, “Sequence to sequence learning with neural networks,” in Adv. Neural Inf. Process. Syst. 27 , Vancouver, BC, Canada, Dec. 2014, pp. 3104–3112
2014
Cited alongside, same era.
U. Mori, A. Mendiburu, M. Álvarez, and J. A. Lozano, “A review of travel time estimation and forecasting for advanced traveller information systems,” Transportmetrica A , vol. 11, no. 2, pp. 119–157, 2015
2015
Cited alongside, same era.
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in 2015 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Boston, MA, USA, Jun. 2015, pp. 3431–3440
2015
Cited alongside, same era.
X. Ma, Z. Tao, Y. Wang, H. Yu, and Y. Wang, “Long short-term memory neural network for traffic speed prediction using remote microwave sensor data,” Transp. Res. Part C Emerg. Technol. , vol. 54, pp. 187–197, May 2015
2015
Cited alongside, same era.
M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” in Adv. Neural Inf. Process. Syst. 29 , Barcelona, Spain, Dec. 2016, pp. 3844–3852
2016
Cited alongside, same era.
Y. Wu and H. Tan, “Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework,” arXiv preprint arXiv:1612.01022 , 2016
Original
2016
Cited alongside, same era.
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in 5th Int. Conf. Learn. Rep. (ICLR) , Toulon, France, Apr. 2017
2017
Cited alongside, same era.
J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin, “Convolutional sequence to sequence learning,” in Proc. 34th Int. Conf. Mach. Learn. , Sydney, NSW, Australia, Aug. 2017, pp. 1243–1252
2017
Cited alongside, same era.
X. Ma et al. , “Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction,” Sensors , vol. 17, no. 4, p. 818, Apr. 2017
2017
Cited alongside, same era.

Then

J. Zhang, Y. Zheng, and D. Qi, “Deep spatio-temporal residual networks for citywide crowd flows prediction,” in Proc. 31st AAAI Conf. Artif. Intell. , San Francisco, CA, USA, Feb. 2017, pp. 1655–1661
2017
Later among the works it cites.
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in Proc. 34th Int. Conf. Mach. Learn. , Sydney, NSW, Australia, Aug. 2017, pp. 1263–1272
2017
Later among the works it cites.
P. Veličković et al. , “Graph attention networks,” in 6th Int. Conf. Learn. Rep. (ICLR) , Vancouver, BC, Canada, May 2017
2017
Later among the works it cites.
A. Vaswani et al. , “Attention is all you need,” in Adv. Neural Inf. Process. Syst. 30 , Long Beach, CA, USA, Dec. 2017, pp. 5998–6008
2017
Later among the works it cites.
B. Yu, H. Yin, and Z. Zhu, “Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting,” in Proc. 27th Int. Joint Conf. Artif. Intell. , Stockholm, Sweden, Jul. 2018, pp. 3634–3640
2018
Later among the works it cites.
Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” in 6th Int. Conf. Learn. Rep. (ICLR) , Vancouver, BC, Canada, May 2018
2018
Later among the works it cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. 2019 Conf. NAACL HLT , Minneapolis, MN, USA, Jun. 2018, pp. 4171–4186
2018
Later among the works it cites.
J. Zhang, Y. Zheng, J. Sun, and D. Qi, “Flow prediction in spatio-temporal networks based on multitask deep learning,” 2019
2019
Later among the works it cites.
S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,” in Proc. 33rd AAAI Conf. Artif. Intell. , Honolulu, HI, USA, Jan. 2019, pp. 922–929
2019
Later among the works it cites.
Z. Pan et al. , “Urban traffic prediction from spatio-temporal data using deep meta learning,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min. , Anchorage, AK, USA, Aug. 2019, pp. 1720–1730
2019
Later among the works it cites.
Z. Diao, X. Wang, D. Zhang, Y. Liu, K. Xie, and S. He, “Dynamic spatial-temporal graph convolutional neural networks for traffic forecasting,” in Proc. 33rd AAAI Conf. Artif. Intell. , Hononlulu, HI, USA, Jan. 2019, pp. 890–897
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…