What does BERT learn about prosody?

J. J. Godfrey, E. C. Holliman, and J. McDaniel, “Switchboard: Telephone speech corpus for research and development,” in Acoustics, Speech, and Signal Processing, IEEE International Conference on , vol. 1. IEEE Computer Society, 1992, pp. 517–520

1992

Earlier work this paper cites.

M. Ostendorf, P. J. Price, and S. Shattuck-Hufnagel, “The boston university radio news corpus,” Linguistic Data Consortium , pp. 1–19, 1995

1995

Earlier work this paper cites.

J. Terken and D. Hermes, “The perception of prosodic prominence,” in Prosody: Theory and experiment . Springer, 2000, pp. 89–127

2000

Earlier work this paper cites.

M. Wagner and D. G. Watson, “Experimental and theoretical advances in prosody: A review,” Language and cognitive processes , vol. 25, no. 7-9, pp. 905–945, 2010

2010

Earlier work this paper cites.

S. Calhoun, J. Carletta, J. M. Brenier, N. Mayo, D. Jurafsky, M. Steedman, and D. Beaver, “The nxt-format switchboard corpus: a rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue,” Language resources and evaluation , vol. 44, no. 4, pp. 387–419, 2010

2010

Earlier work this paper cites.

S. Kakouros and O. Räsänen, “Automatic detection of sentence prominence in speech using predictability of word-level acoustic features,” in Sixteenth Annual Conference of the International Speech Communication Association , 2015

2015

Earlier work this paper cites.

V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2015, pp. 5206–5210

2015

Earlier work this paper cites.

S. Kakouros and O. Räsänen, “Perception of sentence stress in speech correlates with the temporal unpredictability of prosodic features,” Cognitive science , vol. 40, no. 7, pp. 1739–1774, 2016

2016

Earlier work this paper cites.

S. Kakouros, J. Pelemans, L. Verwimp, P. Wambacq, and O. Räsänen, “Analyzing the contribution of top-down lexical and bottom-up acoustic cues in the detection of sentence prominence,” Proceedings Interspeech 2016 , vol. 8, pp. 1074–1078, 2016

2016

Earlier work this paper cites.

S. Kakouros and O. Räsänen, “3pro–an unsupervised method for the automatic detection of sentence prominence in speech,” Speech Communication , vol. 82, pp. 67–84, 2016

2016

Cited alongside, same era.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

2017

Cited alongside, same era.

M. Honnibal and I. Montani, “spaCy 3: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing,” 2017, to appear

2017

Cited alongside, same era.

N. Kitaev and D. Klein, “Constituency parsing with a self-attentive encoder,” arXiv preprint arXiv:1805.01052 , 2018

Original

2018

Cited alongside, same era.

G. Jawahar, B. Sagot, and D. Seddah, “What does bert learn about the structure of language?” in ACL 2019-57th Annual Meeting of the Association for Computational Linguistics , 2019

2019

Later among the works it cites.

I. Tenney, D. Das, and E. Pavlick, “Bert rediscovers the classical nlp pipeline,” arXiv preprint arXiv:1905.05950 , 2019

Original

2019

Later among the works it cites.

I. Tenney, P. Xia, B. Chen, A. Wang, A. Poliak, R. T. McCoy, N. Kim, B. Van Durme, S. R. Bowman, D. Das et al. , “What do you learn from context? probing for sentence structure in contextualized word representations,” arXiv preprint arXiv:1905.06316 , 2019

Original

2019

Later among the works it cites.

K. Clark, U. Khandelwal, O. Levy, and C. D. Manning, “What does bert look at? an analysis of bert’s attention,” arXiv preprint arXiv:1906.04341 , 2019

Original

2019

Later among the works it cites.