Are Neighbors Enough? Multi-Head Neural n-gram can be Alternative to Self-attention

Generating long sequences with sparse transformers

Original

Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. 2019 · 1904

Earlier work this paper cites.

A neural probabilistic language model

Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003 · 2003

Earlier work this paper cites.

Longformer: The long-document transformer

Original

Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020 · 2004

Earlier work this paper cites.

Neural Machine Translation by Jointly Learning to Align and Translate

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015 · 2015

Earlier work this paper cites.

Librispeech: An asr corpus based on public domain audio books

Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015 · 2015

Earlier work this paper cites.

A neural attention model for abstractive sentence summarization

Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015 · 2015

Earlier work this paper cites.

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016 · 2016

Cited alongside, same era.

Neural machine translation of rare words with subword units

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016

Cited alongside, same era.

Get to the point: Summarization with pointer-generator networks

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017 · 2017

Cited alongside, same era.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Cited alongside, same era.

Speech-transformer: A no-recurrence sequence-to-sequence model for speech recognition

Linhao Dong, Shuang Xu, and Bo Xu. 2018 · 2018

Cited alongside, same era.

Classical structured prediction losses for sequence to sequence learning

Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018 · 2018

Later among the works it cites.

SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing

Taku Kudo and John Richardson. 2018 · 2018

Later among the works it cites.

Scaling neural machine translation

Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018 · 2018

Later among the works it cites.

Positional encoding to control output sequence length

Sho Takase and Naoaki Okazaki. 2019 · 2019

Later among the works it cites.

Revisiting simple neural probabilistic language models

Simeng Sun and Mohit Iyyer. 2021 · 2021

Later among the works it cites.