Fetching the paper…
Reading the bibliography…
We present a Three-level Hierarchical Transformer Network (3-level-HTN) for modeling long-term dependencies across clinical notes for the purpose of patient-level prediction.
DocBERT: BERT for document classification
Ashutosh Adhikari, Achyudh Ram, Raphael Tang, and Jimmy Lin. 2019 · 1904
Earlier work this paper cites.
RoBERTa: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Well-read students learn better: On the importance of pre-training compact models
Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 1908
Earlier work this paper cites.
AlBERT: A lite bert for self-supervised learning of language representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019 · 1909
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 1910
Earlier work this paper cites.
Lessons Learned from Applying off-the-shelf BERT: There is no SilverBullet
Victor Makarenkov and Lior Rokach. 2020 · 2009
Earlier work this paper cites.
MIMIC-III, a freely accessible critical care database
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016 · 2016
Earlier work this paper cites.
Hierarchical attention networks for document classification
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016 · 2016
Earlier work this paper cites.
Attention-based LSTM network for cross-lingual sentiment classification
Xinjie Zhou, Xiaojun Wan, and Jianguo Xiao. 2016 · 2016
Earlier work this paper cites.
HDLTex: Hierarchical deep learning for text classification
Kamran Kowsari, Donald E Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S Gerber, and Laura E Barnes. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Hierarchical convolutional attention networks for text classification
Shang Gao, Arvind Ramanathan, and Georgia Tourassi. 2018 · 2018
Cited alongside, same era.
Deep ehr: Chronic disease prediction using medical notes
Jingshu Liu, Zachariah Zhang, and Narges Razavian. 2018 · 2018
Cited alongside, same era.
Training tips for the transformer model
Martin Popel and Ondřej Bojar. 2018 · 2018
Cited alongside, same era.
Attend and diagnose: Clinical time series analysis using attention models
Huan Song, Deepta Rajan, Jayaraman Thiagarajan, and Andreas Spanias. 2018 · 2018
Cited alongside, same era.
Publicly Available Clinical BERT Embeddings
Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott. 2019 · 2019
Cited alongside, same era.
SciBERT: A Pretrained Language Model for Scientific Text
Iz Beltagy, Kyle Lo, and Arman Cohan. 2019 · 2019
Learning the graphical structure of electronic health records with graph convolutional transformer
Edward Choi, Zhen Xu, Yujia Li, Michael Dusenberry, Gerardo Flores, Emily Xue, and Andrew Dai. 2020 · 2020
Later among the works it cites.
Explainable clinical decision support from text
Jinyue Feng, Chantal Shaib, and Frank Rudzicz. 2020 · 2020
Later among the works it cites.
SpanBERT: Improving pre-training by representing and predicting spans
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. 2020 · 2020
Later among the works it cites.
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020 · 2020
Later among the works it cites.
ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network
Fei Li and Hong Yu. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Hierarchical transformers for long document classification
Raghavendra Pappagari, Piotr Zelasko, Jesús Villalba, Yishay Carmiel, and Najim Dehak. 2019 · 2019
Cited alongside, same era.
Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets
Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019 · 2019
Cited alongside, same era.
Enhancing clinical concept extraction with contextual embeddings
Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019 · 2019
Cited alongside, same era.
HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization
Xingxing Zhang, Furu Wei, and Ming Zhou. 2019 · 2019
Cited alongside, same era.
BEHRT: transformer for electronic health records
Yikuan Li, Shishir Rao, José Roberto Ayala Solares, Abdelaali Hassaine, Rema Ramakrishnan, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi-Khorshidi. 2020 · 2020
Later among the works it cites.
Patient Representation Transfer Learning from Clinical Notes based on Hierarchical Attention Network
Yuqi Si and Kirk Roberts. 2020 · 2020
Later among the works it cites.
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Long-Form Document Matching
Liu Yang, Mingyang Zhang, Cheng Li, Michael Bendersky, and Marc Najork. 2020 · 2020
Later among the works it cites.
Big bird: Transformers for longer sequences
Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020 · 2020
Later among the works it cites.
Time-Aware Transformer-based Network for Clinical Notes Series Prediction
Dongyu Zhang, Jidapa Thadajarassiri, Cansu Sen, and Elke Rundensteiner. 2020 · 2020
Later among the works it cites.
Deep representation learning of patient data from electronic health records (ehr): A systematic review
Yuqi Si, Jingcheng Du, Zhao Li, Xiaoqian Jiang, Timothy Miller, Fei Wang, W. Jim Zheng, and Kirk Roberts. 2021 · 2021
Closest in time.