Fetching the paper…
Reading the bibliography…
Pre-trained transformers are often fine-tuned to aid clinical decision-making using limited clinical notes.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov · 1907
Earlier work this paper cites.
Publicly available clinical BERT embeddings
Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott · 1909
Earlier work this paper cites.
Domain-specific language model pretraining for biomedical natural language processing
Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon · 1957
Earlier work this paper cites.
Content and cluster analysis: Assessing representational similarity in neural systems
Aarre Laakso and G. Cottrell · 2000
Earlier work this paper cites.
Unilmv2: Pseudo-masked language models for unified language model pre-training
Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Songhao Piao, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon · 2002
Earlier work this paper cites.
Laila Rasmy, Yang Xiang, Ziqian Xie, Cui Tao, and Degui Zhi · 2005
Earlier work this paper cites.
Representational similarity analysis - connecting the branches of systems neuroscience
Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini · 2008
Earlier work this paper cites.
Aligning books and movies: Towards story-like visual explanations by watching movies and reading books
Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler · 2015
Earlier work this paper cites.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean · 2016
Earlier work this paper cites.
Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Insights on representational similarity in neural networks with canonical correlation
Ari S. Morcos, Maithra Raghu, and Samy Bengio · 2018
Earlier work this paper cites.
Dissecting contextual word embeddings: Architecture and representation
Matthew E. Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih · 2018
Earlier work this paper cites.
A simple method for commonsense reasoning
Trieu H. Trinh and Quoc V. Le · 2018
Earlier work this paper cites.
Language modeling teaches you more than translation does: Lessons learned through auxiliary syntactic task analysis
Kelly Zhang and Samuel Bowman · 2018
Earlier work this paper cites.
Blackbox meets blackbox: Representational similarity & stability analysis of neural language models and brains
Samira Abnar, Lisa Beinborn, Rochelle Choenni, and Willem Zuidema · 2019
Earlier work this paper cites.
Correlating neural and symbolic representations of language
Grzegorz Chrupała and Afra Alishahi · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Linking artificial and human neural representations of language
Jon Gauthier and R. Levy · 2019
Cited alongside, same era.
Designing and interpreting probes with control tasks
John Hewitt and Percy Liang · 2019
Cited alongside, same era.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D. Manning · 2019
Cited alongside, same era.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
What happens to BERT embeddings during fine-tuning?
Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, and Ian Tenney · 2020
Later among the works it cites.
Natural language processing systems for pathology parsing in limited data environments with uncertainty estimation
Anobel Y Odisho, Briton Park, Nicholas Altieri, John DeNero, Matthew R Cooperberg, Peter R Carroll, and Bin Yu · 2020
Later among the works it cites.
exbert: Extending pre-trained models with domain-specific vocabulary under constrained training resources
Wen-Hsin Tai, H. T. Kung, and Xin Dong · 2020
Later among the works it cites.
srbert: automatic article classification model for systematic review using bert
Sungmin Aum and Seon Choe · 2021
Later among the works it cites.
Limitations of transformers on clinical text classification
Shang Gao, Mohammed M. Alawad, Michael T. Young, John P. Gounley, Noah Schaefferkoetter, Hong-Jun Yoon, Xiao-Cheng Wu, Eric B. Durbin, Jennifer Anne Doherty, Antoinette M. Stroup, Linda Coyle, and Georgia D. Tourassi · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang · 2019
Cited alongside, same era.
Fine-tuning bidirectional encoder representations from transformers (bert)–based models on large-scale electronic health record notes: An empirical study
Fei Li, Yonghao Jin, Weisong Liu, Bhanu Pratap Singh Rawat, Pengshan Cai, and Hong Yu · 2019
Cited alongside, same era.
Linguistic knowledge and transferability of contextual representations
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith · 2019
Cited alongside, same era.
To tune or not to tune? adapting pretrained representations to diverse tasks
Matthew E. Peters, Sebastian Ruder, and Noah A. Smith · 2019
Cited alongside, same era.
Understanding learning dynamics of language models with SVCCA
Naomi Saphra and Adam Lopez · 2019
Cited alongside, same era.
What do you learn from context? probing for sentence structure in contextualized word representations
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick · 2019
Cited alongside, same era.
How does bert answer questions? a layer-wise analysis of transformer representations
Betty van Aken, Benjamin Winter, Alexander Löser, and Felix A. Gers · 2019
Cited alongside, same era.
The classification of short scientific texts using pretrained bert model
J Mantas et al · 2021
Later among the works it cites.
Automatic extraction of 12 cardiovascular concepts from german discharge letters using pre-trained language models
Phillip Richter-Pechanski, Nicolas A. Geis, Christina Kiriakou, Dominic M. Schwab, and Christoph Dieterich · 2021
Later among the works it cites.
Leveraging pretrained models for automatic summarization of doctor-patient conversations
Longxiang Zhang, Renato Negrinho, Arindam Ghosh, Vasudevan Jagannathan, Hamid Reza Hassanzadeh, Thomas Schaaf, and Matthew R. Gormley · 2021
Later among the works it cites.
Using bert to identify drug-target interactions from whole pubmed
Jehad Aldahdooh, Markus Vähä-Koskela, Jing Tang, and Ziaurrehman Tanoli · 2022
Later among the works it cites.
In-domain pre-training improves clinical note generation from doctor-patient conversations
Colin Grambow, Longxiang Zhang, and Thomas Schaaf · 2022
Later among the works it cites.
Critical assessment of transformer-based AI models for German clinical notes
Manuel Lentzen, Sumit Madan, Vanessa Lage-Rupprecht, Lisa Kühnel, Juliane Fluck, Marc Jacobs, Mirja Mittermaier, Martin Witzenrath, Peter Brunecker, Martin Hofmann-Apitius, Joachim Weber, and Holger Fröhlich · 2022
Later among the works it cites.
Sam Preston, Mu-Hsin Wei, Rajesh Rao, Robert Tinn, Naoto Usuyama, Michael R. Lucas, Yu Gu, Roshanthi Weerasinghe, Soo Youl Lee, Brian D. Piening, Paul D. Tittel, Naveen Valluri, Tristan Naumann, Carlo B. Bifulco, and Hoifung Poon · 2022
Later among the works it cites.
Performance of multiple pretrained bert models to automate and accelerate data annotation for large datasets
Ali S Tejani, Yee S Ng, Yin Xi, Julia R Fielding, Travis G Browning, and Jesse C Rayan · 2022
Later among the works it cites.
Abstractive summarization of hospitalisation histories with transformer networks
Alexander Yalunin, Dmitriy Umerenkov, and Vladimir Kokh · 2022
Later among the works it cites.
Primarygleasonbert
Jenna Kefeli and Nicholas Tatonetti · 2023
Closest in time.
Fine-tuning large neural language models for biomedical natural language processing
Robert Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon · 2023
Closest in time.