Fetching the paper…
Reading the bibliography…
Deep learning has recently made remarkable progress in natural language processing.
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov · 1901
Earlier work this paper cites.
The emergence of number and syntax units in LSTM language models
Yair Lakretz, German Kruszewski, Theo Desbordes, Dieuwke Hupkes, Stanislas Dehaene, and Marco Baroni · 1903
Earlier work this paper cites.
The Curious Case of Neural Text Degeneration
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi · 1904
Earlier work this paper cites.
Linguistic generalization and compositionality in modern artificial neural networks
Marco Baroni · 1904
Earlier work this paper cites.
Mariya Toneva and Leila Wehbe · 1905
Earlier work this paper cites.
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le · 1906
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov · 1907
Earlier work this paper cites.
Tracking Naturalistic Linguistic Predictions with Deep Neural Language Models
Micha Heilbron, Benedikt Ehinger, Peter Hagoort, and Floris P. de Lange · 1909
Earlier work this paper cites.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 1910
Earlier work this paper cites.
Momentum Contrast for Unsupervised Visual Representation Learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick · 1911
Earlier work this paper cites.
Using stochastic language models (SLM) to map lexical, syntactic, and phonological information processing in the brain
Alessandro Lopopolo, Stefan L. Frank, Antal van den Bosch, and Roel M. Willems · 1932
Earlier work this paper cites.
An interactive activation model of context effects in letter perception: II. The contextual enhancement effect and some tests and extensions of the model
David E. Rumelhart and James L. McClelland · 1939
Earlier work this paper cites.
An interactive activation model of context effects in letter perception: I. An account of basic findings
James L. McClelland and David E. Rumelhart · 1939
Earlier work this paper cites.
Deficits in strategy application following frontal lobe damage in man
Tim Shallice and Paul Burgess · 1991
Earlier work this paper cites.
Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects
Rajesh P. N. Rao and Dana H. Ballard · 1999
Earlier work this paper cites.
Foundations of language: Brain, meaning, grammar, evolution
Ray Jackendoff and Ray S Jackendoff · 2002
Earlier work this paper cites.
A Simple Framework for Contrastive Learning of Visual Representations
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton · 2002
Earlier work this paper cites.
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2005
Earlier work this paper cites.
Executive function
Sam J. Gilbert and Paul W. Burgess · 2007
Earlier work this paper cites.
Word meaning in minds and machines
Brenden M. Lake and Gregory L. Murphy · 2008
Earlier work this paper cites.
Predictive coding under the free-energy principle
Karl Friston and Stefan Kiebel · 2008
Earlier work this paper cites.
Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature
Christophe Destrieux, Bruce Fischl, Anders Dale, and Eric Halgren · 2010
Earlier work this paper cites.
Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story
Y. Lerner, C. J. Honey, L. J. Silbert, and U. Hasson · 2011
Cited alongside, same era.
Scikit-learn: Machine learning in Python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay · 2011
Cited alongside, same era.
Aligning context-based statistical models of language with brain activity during reading
Leila Wehbe, Ashish Vaswani, Kevin Knight, and Tom Mitchell · 2014
Cited alongside, same era.
Performance-optimized hierarchical models predict neural responses in higher visual cortex
D. L. K. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. J. DiCarlo · 2014
Cited alongside, same era.
Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation
Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte · 2014
Cited alongside, same era.
fMRIPrep: a robust preprocessing pipeline for functional MRI
Oscar Esteban, Christopher J. Markiewicz, Ross W. Blair, Craig A. Moodie, A. Ilkay Isik, Asier Erramuzpe, James D. Kent, Mathias Goncalves, Elizabeth DuPre, Madeleine Snyder, Hiroyuki Oya, Satrajit S. Ghosh, Jessey Wright, Joke Durnez, Russell A. Poldrack, and Krzysztof J. Gorgolewski · 2019
Later among the works it cites.
Language processing in brains and deep neural networks: computational convergence and its limits
Charlotte Caucheteux and Jean-Rémi King · 2020
Later among the works it cites.
Artificial Neural Networks Accurately Predict Language Processing in the Brain
Martin Schrimpf, Idan Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua Tenenbaum, and Evelina Fedorenko · 2020
Later among the works it cites.
Thinking ahead: prediction in context as a keystone of language in humans and machines
Ariel Goldstein, Zaid Zada, Eliav Buchnik, Mariano Schain, Amy Price, Bobbi Aubrey, Samuel A. Nastase, Amir Feder, Dotan Emanuel, Alon Cohen, Aren Jansen, Harshvardhan Gazula, Gina Choe, Aditi Rao, Catherine Kim, Colton Casto, Fanda Lora, Adeen Flinker, Sasha Devore, Werner Doyle, Patricia Dugan, Daniel Friedman, Avinatan Hassidim, Michael Brenner, Yossi Matias, Ken A. Norman, Orrin Devinsky, and Uri Hasson · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream
Umut Güçlü and Marcel A. J. van Gerven · 2015
Cited alongside, same era.
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2015
Cited alongside, same era.
Prediction During Natural Language Comprehension
Roel M. Willems, Stefan L. Frank, Annabel D. Nijhof, Peter Hagoort, and Antal van den Bosch · 2016
Cited alongside, same era.
Natural speech reveals the semantic maps that tile human cerebral cortex
Alexander G. Huth, Wendy A. de Heer, Thomas L. Griffiths, Frédéric E. Theunissen, and Jack L. Gallant · 2016
Cited alongside, same era.
Seeing it all: Convolutional network layers map the function of the human visual system
Michael Eickenberg, Alexandre Gramfort, Gaël Varoquaux, and Bertrand Thirion · 2016
Cited alongside, same era.
Cortical tracking of hierarchical linguistic structures in connected speech
Nai Ding, Lucia Melloni, Hang Zhang, Xing Tian, and David Poeppel · 2016
Cited alongside, same era.
Attention is All you Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Neural language models capture some, but not all, agreement attraction effects
Suhas Arehalli and Tal Linzen · 2020
Later among the works it cites.
Gpt-2 and the nature of intelligence
Gary Marcus · 2020
Later among the works it cites.
Brain signatures of surprise in EEG and MEG data
Zahra Mousavi, Mohammad Mahdi Kiani, and Hamid Aghajan · 2020
Later among the works it cites.
Narratives: fMRI data for evaluating models of naturalistic language comprehension
Samuel A. Nastase, Yun-Fei Liu, Hanna Hillman, Asieh Zadbood, Liat Hasenfratz, Neggin Keshavarzian, Janice Chen, Christopher J. Honey, Yaara Yeshurun, Mor Regev, Mai Nguyen, Claire H. C. Chang, Christopher Baldassano, Olga Lositsky, Erez Simony, Michael A. Chow, Yuan Chang Leong, Paula P. Brooks, Emily Micciche, Gina Choe, Ariel Goldstein, Tamara Vanderwal, Yaroslav O. Halchenko, Kenneth A. Norman, and Uri Hasson · 2020
Later among the works it cites.
The meaning that emerges from combining words is robustly localizable in space but not in time
Mariya Toneva, Tom M. Mitchell, and Leila Wehbe · 2020
Later among the works it cites.
ELECTRA: PRE-TRAINING TEXT ENCODERS AS DISCRIMINATORS RATHER THAN GENERATORS
Kevin Clark, Minh-Thang Luong, and Quoc V Le · 2020
Later among the works it cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2020
Later among the works it cites.
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors · 2020
Later among the works it cites.
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych · 2021
Closest in time.
Hurdles to Progress in Long-form Question Answering
Kalpesh Krishna, Aurko Roy, and Mohit Iyyer · 2021
Closest in time.
Can RNNs learn Recursive Nested Subject-Verb Agreements?
Yair Lakretz, Théo Desbordes, Jean-Rémi King, Benoît Crabbé, Maxime Oquab, and Stanislas Dehaene · 2021
Closest in time.
GPT-2’s activations predict the degree of semantic comprehension in the human brain
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King · 2021
Closest in time.
Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King · 2021
Closest in time.
‘constituent length’ effects in fmri do not provide evidence for abstract syntactic processing
Cory Shain, Hope Kean, Benjamin Lipkin, Josef Affourtit, Matthew Siegelman, Francis Mollica, and Evelina Fedorenko · 2021
Closest in time.
XCiT: Cross-Covariance Image Transformers
Alaaeldin El-Nouby, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, and Hervé Jegou · 2021
Closest in time.
Vicreg: Variance-invariance-covariance regularization for self-supervised learning
Adrien Bardes, Jean Ponce, and Yann LeCun · 2021
Closest in time.
Language prediction mechanisms in human auditory cortex
K. J. Forseth, G. Hickok, P. S. Rollo, and N. Tandon · 2041
Closest in time.
Anticipation of temporally structured events in the brain
Caroline S. Lee, Mariam Aly, and Christopher Baldassano · 2050
Closest in time.