Fetching the paper…
Reading the bibliography…
The number of Language Models (LMs) dedicated to processing scientific text is on the rise.
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2020a · 1904
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
FinBERT: Financial Sentiment Analysis with Pre-trained Language Models
Dogu Araci. 2019 · 1908
Earlier work this paper cites.
BioFLAIR: Pretrained Pooled Contextualized Embeddings for Biomedical Sequence Labeling Tasks
Shreyas Sharma and Ron Daniel Jr au2. 2019 · 1908
Earlier work this paper cites.
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020 · 1909
Earlier work this paper cites.
Clinical concept extraction using transformers
Xi Yang, Jiang Bian, William R Hogan, and Yonghui Wu. 2020 · 1942
Earlier work this paper cites.
MacBERTh: Development and Evaluation of a Historically Pre-trained Language Model for English (1450-1950). In Workshop on Natural Language Processing for Digital Humanities . 23–36
Enrique Manjavacas Arevalo and Lauren Fonteyn. 2021 · 1950
Earlier work this paper cites.
Statistical inference for probabilistic functions of finite state Markov chains
Leonard E Baum and Ted Petrie. 1966 · 1966
Earlier work this paper cites.
Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel
J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom. 1975 · 1975
Earlier work this paper cites.
SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules
David Weininger. 1988 · 1988
Earlier work this paper cites.
Class-Based n
Peter F. Brown, Vincent J. Della Pietra, Peter V. deSouza, Jenifer C. Lai, and Robert L. Mercer. 1992 · 1992
Earlier work this paper cites.
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In ICML . 282–289
John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001 · 2001
Earlier work this paper cites.
NukeBERT: A Pre-trained language model for Low Resource Nuclear Domain
Ayush Jain, Dr. N. M. Meenachi, and Dr. B. Venkatraman. 2020 · 2003
Earlier work this paper cites.
Longformer: The Long-Document Transformer
Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020 · 2004
Earlier work this paper cites.
Introduction to the Bio-entity Recognition Task at JNLPBA. In NLPBA/BioNLP . 73–78
Nigel Collier, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Jin-Dong Kim. 2004 · 2004
Earlier work this paper cites.
The introduction, methods, results, and discussion (IMRAD) structure: a fifty-year survey
Luciana B Sollaci and Mauricio G Pereira. 2004 · 2004
Earlier work this paper cites.
Pre-training technique to localize medical BERT and enhance biomedical BERT
Shoya Wada, Toshihiro Takeda, Shiro Manabe, Shozo Konishi, Jun Kamohara, and Yasushi Matsumura. 2021 · 2005
Earlier work this paper cites.
Overview of BioCreative II gene mention recognition
Larry Smith, Lorraine K Tanabe, Rie Johnson nee Ando, Cheng-Ju Kuo, I-Fang Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, Christoph M Friedrich, Kuzman Ganchev, et al · 2008
Earlier work this paper cites.
Conceptualized Representation Learning for Chinese Biomedical Text Mining
Ningyu Zhang, Qianghuai Jia, Kangping Yin, Liang Dong, Feng Gao, and Nengwei Hua. 2020a · 2008
Earlier work this paper cites.
Natural language processing with Python: analyzing text with the natural language toolkit
Steven Bird, Ewan Klein, and Edward Loper. 2009 · 2009
Earlier work this paper cites.
The ACL Anthology Network Corpus. In NLPIR4DL . 54–61
Dragomir R. Radev, Pradeep Muthukrishnan, and Vahed Qazvinian. 2009 · 2009
Earlier work this paper cites.
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
Seyone Chithrananda, Gabriel Grand, and Bharath Ramsundar. 2020 · 2010
Earlier work this paper cites.
LINNAEUS: A species name identification system for biomedical literature
Martin Gerner, G. Nenadic, and Casey M. Bergman. 2010 · 2010
Earlier work this paper cites.
Language Models are Open Knowledge Graphs
Chenguang Wang, Xiao Liu, and Dawn Song. 2020 · 2010
Earlier work this paper cites.
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011 · 2011
Earlier work this paper cites.
Academic and scientific texts: the same or different communities?
David R Russell and Viviana Cortes. 2012 · 2012
Earlier work this paper cites.
The EU-ADR corpus: Annotated drugs, diseases, targets, and their relationships
Erik M. van Mulligen, Annie Fourrier-Reglat, David Gurwitz, Mariam Molokhia, Ainhoa Nieto, Gianluca Trifiro, Jan A. Kors, and Laura I. Furlong. 2012 · 2012
Earlier work this paper cites.
Translating embeddings for modeling multi-relational data
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013 · 2013
Earlier work this paper cites.
The DDI corpus: An annotated corpus with pharmacological substances and drug–drug interactions
María Herrero-Zazo, Isabel Segura-Bedmar, Paloma Martínez, and Thierry Declerck. 2013 · 2013
Earlier work this paper cites.
Efficient Estimation of Word Representations in Vector Space. In ICLR
Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 · 2013
Earlier work this paper cites.
The SPECIES and ORGANISMS resources for fast and accurate identification of taxonomic names in text
Evangelos Pafilis, Sune Frankild, Lucia Fanini, Sarah Faulwetter, Christina Pavloudi, Katerina Vasileiadou, C. Arvanitidis, and Lars Jensen. 2013 · 2013
Earlier work this paper cites.
Evaluating temporal relations in clinical text: 2012 i2b2 Challenge
Weiyi Sun, Anna Rumshisky, and Özlem Uzuner. 2013 · 2013
Earlier work this paper cites.
CiteSeerx: A Scholarly Big Dataset. In Advances in Information Retrieval . 311–322
Cornelia Caragea, Jian Wu, Alina Ciobanu, Kyle Williams, Juan Fernández-Ramírez, Hung-Hsuan Chen, Zhaohui Wu, and Lee Giles. 2014 · 2014
Earlier work this paper cites.
NCBI disease corpus: A resource for disease name recognition and concept normalization
Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014 · 2014
Earlier work this paper cites.
Extraction of relations between genes and diseases from text and large-scale data analysis: Implications for translational research
Àlex Bravo, Janet Piñero, Núria Queralt-Rosinach, Michael Rautschka, and Laura I Furlong. 2015 · 2015
Earlier work this paper cites.
A Neural Probabilistic Model for Context Based Citation Recommendation. In AAAI
Wenyi Huang, Zhaohui Wu, Chen Liang, Prasenjit Mitra, and C. Lee Giles. 2015 · 2015
Earlier work this paper cites.
CHEMDNER: The drugs and chemical names extraction challenge
Martin Krallinger, Florian Leitner, Obdulia Rabal, Miguel Vazquez, Julen Oyarzabal, and Alfonso Valencia. 2015 · 2015
Earlier work this paper cites.
Incorporating domain knowledge in chemical and biomedical named entity recognition with word representations
Tsendsuren Munkhdalai, Meijing Li, Khuyagbaatar Batsuren, Hyeon Ah Park, Nak Hyeon Choi, and Keun Ho Ryu. 2015 · 2015
Earlier work this paper cites.
An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition
George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R Alvers, Dirk Weissenborn, et al · 2015
Earlier work this paper cites.
Character-level convolutional networks for text classification
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015 · 2015
Earlier work this paper cites.
Long Short-Term Memory-Networks for Machine Reading. In EMNLP . 551–561
Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016 · 2016
Earlier work this paper cites.
ChemProt-3.0: a global chemical biology diseases mapping
Jens Kringelum, Sonny Kim Kjaerulff, Søren Brunak, Ole Lund, Tudor I. Oprea, and Olivier Taboureau. 2016 · 2016
Earlier work this paper cites.
BioCreative V CDR task corpus: a resource for chemical disease relation extraction
Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016 · 2016
Earlier work this paper cites.
Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al · 2016
Earlier work this paper cites.
Representation Learning on Graphs: Methods and Applications
William L. Hamilton, Rex Ying, and Jure Leskovec. 2017 · 2017
Earlier work this paper cites.
A Structured Self-Attentive Sentence Embedding. In ICLR
Zhouhan Lin, Minwei Feng, Cícero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
QuAC: Question Answering in Context. In EMNLP . 2174–2184
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018 · 2018
Earlier work this paper cites.
Simple and Effective Semi-Supervised Question Answering. In NAACL . 582–587
Bhuwan Dhingra, Danish Danish, and Dheeraj Rajagopal. 2018 · 2018
Earlier work this paper cites.
Measuring the Evolution of a Scientific Field through Citation Frames
David Jurgens, Srijan Kumar, Raine Hoover, Dan McFarland, and Dan Jurafsky. 2018 · 2018
Earlier work this paper cites.
A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature. In ACL . 197–207
Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain Marshall, Ani Nenkova, and Byron Wallace. 2018 · 2018
Earlier work this paper cites.
emrQA: A Large Corpus for Question Answering on Electronic Medical Records. In EMNLP . 2357–2368
Anusri Pampari, Preethi Raghavan, Jennifer Liang, and Jian Peng. 2018 · 2018
Earlier work this paper cites.
BioRead: A New Dataset for Biomedical Reading Comprehension. In LREC
Dimitris Pappas, Ion Androutsopoulos, and Haris Papageorgiou. 2018 · 2018
Earlier work this paper cites.
Deep Contextualized Word Representations. In NAACL . 2227–2237
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018
Earlier work this paper cites.
Improving Language Understanding by Generative Pre-Training
Alec Radford and Karthik Narasimhan. 2018 · 2018
Earlier work this paper cites.
Semantically Equivalent Adversarial Rules for Debugging NLP models. In ACL
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018 · 2018
Earlier work this paper cites.
Lessons from Natural Language Inference in the Clinical Domain. In EMNLP . 1586–1596
Alexey Romanov and Chaitanya Shivade. 2018 · 2018
Earlier work this paper cites.
CL Scholar: The ACL Anthology Knowledge Graph Miner. In NAACL . 16–20
Mayank Singh, Pradeep Dogga, Sohan Patro, Dhiraj Barnwal, Ritam Dutt, Rajarshi Haldar, Pawan Goyal, and Animesh Mukherjee. 2018 · 2018
Earlier work this paper cites.
Constructing Datasets for Multi-hop Reading Comprehension Across Documents
Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018 · 2018
Earlier work this paper cites.
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In EMNLP . 2369–2380
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018 · 2018
Earlier work this paper cites.
FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP. In NAACL (Demonstrations) . 54–59
Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019 · 2019
Earlier work this paper cites.
Publicly Available Clinical BERT Embeddings. In Clinical Natural Language Processing Workshop . 72–78
Emily Alsentzer, John Murphy, William Boag, Wei-Hung Weng, Di Jindi, Tristan Naumann, and Matthew McDermott. 2019 · 2019
Earlier work this paper cites.
SciBERT: A Pretrained Language Model for Scientific Text. In EMNLP-IJCNLP . 3615–3620
Iz Beltagy, Kyle Lo, and Arman Cohan. 2019 · 2019
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL . 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Visualizing and Understanding the Effectiveness of BERT. In EMNLP-IJCNLP . 4143–4152
Yaru Hao, Li Dong, Furu Wei, and Ke Xu. 2019 · 2019
Earlier work this paper cites.
Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA. In ACL . 2726–2736
Yichen Jiang and Mohit Bansal. 2019 · 2019
Earlier work this paper cites.
Fine-Tuning Bidirectional Encoder Representations From Transformers (BERT)–Based Models on Large-Scale Electronic Health Record Notes: An Empirical Study
Fei Li, Yonghao Jin, Weisong Liu, Bhanu Pratap Singh Rawat, Pengshan Cai, and Hong Yu. 2019 · 2019
Earlier work this paper cites.
The NLP4NLP Corpus (I): 50 Years of Publication, Collaboration and Citation in Speech and Language Processing
Joseph Mariani, Gil Francopoulo, and Patrick Paroubek. 2019 · 2019
Earlier work this paper cites.
Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets. In BioNLP . 58–65
Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019 · 2019
Earlier work this paper cites.
Language Models are Unsupervised Multitask Learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Earlier work this paper cites.
CoQA: A Conversational Question Answering Challenge
Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019 · 2019
Earlier work this paper cites.
Bibliometric-Enhanced arXiv: A Data Set for Paper-Based and Citation-Based Tasks. In BIR@ECIR
Tarek Saier and Michael Färber. 2019 · 2019
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. In EMC2@NeurIPS 2019
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 2019
Earlier work this paper cites.
Pre-training of Graph Augmented Transformers for Medication Recommendation. In IJCAI . 5953–5959
Junyuan Shang, Tengfei Ma, Cao Xiao, and Jimeng Sun. 2019 · 2019
Earlier work this paper cites.
Enhancing clinical concept extraction with contextual embeddings
Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019 · 2019
Earlier work this paper cites.
XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NeurIPS . 5754–5764
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019 · 2019
Earlier work this paper cites.
PubLayNet: Largest Dataset Ever for Document Layout Analysis. In ICDAR . 1015–1022
Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. 2019 · 2019
Earlier work this paper cites.
An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training. In EMNLP . 4854–4864
Kristjan Arumae, Qing Sun, and Parminder Bhatia. 2020 · 2020
Cited alongside, same era.
COMETA: A Corpus for Medical Entity Linking in the Social Media. In EMNLP . 3122–3137
Marco Basaldella, Fangyu Liu, Ehsan Shareghi, and Nigel Collier. 2020 · 2020
Cited alongside, same era.
Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports
Keno K Bressem, Lisa C Adams, Robert A Gaudin, Daniel Troltzsch, Bernd Hamm, Marcus R Makowski, et al · 2020
Cited alongside, same era.
Language Models are Few-Shot Learners. In NeurIPS
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
BioMedBERT: A Pre-trained Biomedical Language Model for QA and IR. In COLING . 669–679
Souradip Chakraborty, Ekaba Bisong, Shweta Bhatt, Thomas Wagner, Riley Elliott, and Francesco Mosconi. 2020 · 2020
Benchmarking for biomedical natural language processing tasks with a domain specific ALBERT
Usman Naseem, Adam Dunn, Matloob Khushi, and Jinman Kim. 2022 · 2022
Later among the works it cites.
A survey of automated methods for biomedical text simplification
Brian Ondov, Kush Attal, and Dina Demner-Fushman. 2022 · 2022
Later among the works it cites.
PetroBERT: A Domain Adaptation Language Model for Oil and Gas Applications in Portuguese. In PROPOR . 101–109
Rafael BM Rodrigues, Pedro IM Privatto, Gustavo José de Sousa, Rafael P Murari, Luis CS Afonso, João P Papa, et al · 2022
Later among the works it cites.
Clinical Flair: A Pre-Trained Language Model for Spanish Clinical Natural Language Processing. In Clinical Natural Language Processing Workshop . 87–92
Matías Rojas, Jocelyn Dunstan, and Fabián Villena. 2022 · 2022
Later among the works it cites.
The Effects of In-domain Corpus Size on pre-training BERT
Chris Sanchez and Zheyuan Zhang. 2022 · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In ICLR
Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020 · 2020
Cited alongside, same era.
SPECTER: Document-level Representation Learning using Citation-informed Transformers. In ACL . 2270–2282
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020 · 2020
Cited alongside, same era.
CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters. In COLING . 6903–6915
Hicham El Boukkouri, Olivier Ferret, Thomas Lavergne, Hiroshi Noji, Pierre Zweigenbaum, and Jun’ichi Tsujii. 2020 · 2020
Cited alongside, same era.
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020 · 2020
Cited alongside, same era.
spaCy: Industrial-strength NLP in Python
Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020 · 2020
Cited alongside, same era.
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. 2020 · 2020
Cited alongside, same era.
Self-referencing embedded strings (SELFIES): A 100% robust molecular string representation
Mario Krenn, Florian Häse, AkshatKumar Nigam, Pascal Friederich, and Alán Aspuru-Guzik. 2020 · 2020
Cited alongside, same era.
Later among the works it cites.
PathologyBERT - Pre-trained Vs. A New Transformer Language Model for Pathology Domain. In AMIA
Thiago Santos, Amara Tariq, Susmita Das, Kavyasree Vayalpati, Geoffrey H. Smith, Hari Trivedi, and Imon Banerjee. 2022 · 2022
Later among the works it cites.
Bloom: A 176b-parameter open-access multilingual language model
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, et al · 2022
Later among the works it cites.
hmBERT: Historical Multilingual Language Models for Named Entity Recognition. In CLEF . 1109–1129
Stefan Schweter, Luisa März, Katharina Schmid, and Erion Çano. 2022 · 2022
Later among the works it cites.
SsciBERT: A Pre-Trained Language Model for Social Science Texts
Si Shen, Jiangfeng Liu, Litao Lin, Ying Huang, Lin Zhang, Chang Liu, Yutong Feng, and Dongbo Wang. 2022a · 2022
Later among the works it cites.
VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups
Zejiang Shen, Kyle Lo, Lucy Lu Wang, Bailey Kuehl, Daniel S. Weld, and Doug Downey. 2022b · 2022
Later among the works it cites.
Developing a general-purpose clinical language inference model from a large corpus of clinical notes
Madhumita Sushil, Dana Ludwig, Atul J. Butte, and Vivek A. Rudrapatna. 2022 · 2022
Later among the works it cites.
Galactica: A Large Language Model for Science
Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022 · 2022
Later among the works it cites.
Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science
Amalie Trewartha, Nicholas Walker, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, et al · 2022
Later among the works it cites.
Bioberturk: Exploring Turkish Biomedical Language Model Development Strategies in Low Resource Setting
Hazal Turkmen, Oguz Dikenelli, Cenk Eraslan, and Mehmet Cem Callı. 2022 · 2022
Later among the works it cites.
D3: A Massive Dataset of Scholarly Metadata for Analyzing the State of Computer Science Research. In LREC . 2642–2651
Jan Philip Wahle, Terry Ruas, Saif Mohammad, and Bela Gipp. 2022 · 2022
Later among the works it cites.
Pre-Trained Language Models and Their Applications
Haifeng Wang, Jiwei Li, Hua Wu, Eduard Hovy, and Yu Sun. 2022a · 2022
Later among the works it cites.
Progress in Machine Translation
Haifeng Wang, Hua Wu, Zhongjun He, Liang Huang, and Kenneth Ward Church. 2022b · 2022
Later among the works it cites.
ClimateBert: A Pretrained Language Model for Climate-Related Text
Nicolas Webersinke, Mathias Kraus, Julia Anna Bingler, and Markus Leippold. 2022 · 2022
Later among the works it cites.
A Japanese Masked Language Model for Academic Domain. In Workshop on Scholarly Document Processing . 152–157
Hiroki Yamauchi, Tomoyuki Kajiwara, Marie Katsurai, Ikki Ohmukai, and Takashi Ninomiya. 2022 · 2022
Later among the works it cites.
RadBERT: Adapting transformer-based language models to radiology
An Yan, Julian McAuley, Xing Lu, Jiang Du, Eric Y Chang, Amilcare Gentili, and Chun-Nan Hsu. 2022 · 2022
Later among the works it cites.
A large language model for electronic health records
Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Anthony B Costa, Mona G Flores, et al · 2022
Later among the works it cites.
MaterialBERT for natural language processing of materials science texts
Michiko Yoshitake, Fumitaka Sato, Hiroyuki Kawano, and Hiroshi Teraoka. 2022 · 2022
Later among the works it cites.
CODER: Knowledge-infused cross-lingual medical term embedding for term normalization
Zheng Yuan, Zhengyun Zhao, Haixia Sun, Jiao Li, Fei Wang, and Sheng Yu. 2022b · 2022
Later among the works it cites.
CancerBERT: a cancer domain-specific language model for extracting breast cancer phenotypes from electronic health records
Sicheng Zhou, Nan Wang, Liwei Wang, Hongfang Liu, and Rui Zhang. 2022 · 2022
Later among the works it cites.
NuclearQA: A Human-Made Benchmark for Language Models for the Nuclear Domain
Anurag Acharya, Sai Munikoti, Aaron Hellinger, Sara Smith, Sridevi Wagle, and Sameera Horawalavithana. 2023 · 2023
Later among the works it cites.
SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model. In AIAA AVIATION 2023 Forum . 3437
Sequoia R Andrade and Hannah S Walsh. 2023 · 2023
Later among the works it cites.
DisorBERT: A Double Domain Adaptation Model for Detecting Signs of Mental Disorders in Social Media. In ACL . 15305–15318
Mario Aragon, Adrian Pastor Lopez Monroy, Luis Gonzalez, David E. Losada, and Manuel Montes. 2023 · 2023
Later among the works it cites.
BioNART: A Biomedical Non-AutoRegressive Transformer for Natural Language Generation. In BioNLP
Masaki Asada and Makoto Miwa. 2023 · 2023
Later among the works it cites.
AliBERT: A Pre-trained Language Model for French Biomedical Text. In BioNLP . 223–236
Aman Berhe, Guillaume Draznieks, Vincent Martenot, Valentin Masdeu, Lucas Davy, and Jean-Daniel Zucker. 2023 · 2023
Later among the works it cites.
Autonomous chemical research with large language models
Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. 2023 · 2023
Later among the works it cites.
ChemCrow: Augmenting large-language models with chemistry tools
Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. 2023 · 2023
Later among the works it cites.
ChestXRayBERT: A Pretrained Language Model for Chest Radiology Report Summarization
Xiaoyan Cai, Sen Liu, Junwei Han, Libin Yang, Zhenguo Liu, and Tianming Liu. 2023 · 2023
Later among the works it cites.
Aviation-BERT: A Preliminary Aviation-Specific Natural Language Model. In AIAA AVIATION 2023 Forum . 3436
Chetan Chandra, Xiao Jing, Mayank V Bendarkar, Kshitij Sawant, Lidya Elias, Michelle Kirby, and Dimitri N Mavris. 2023 · 2023
Later among the works it cites.
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, et al · 2023
Later among the works it cites.
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al · 2023
Later among the works it cites.
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023 · 2023
Later among the works it cites.
EriBERTa: A Bilingual Pre-Trained Language Model for Clinical Natural Language Processing
Iker de la Iglesia, Aitziber Atutxa, Koldo Gojenola, and Ander Barrena. 2023 · 2023
Later among the works it cites.
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
Cheng Deng, Tianhang Zhang, Zhongmou He, Yi Xu, Qiyuan Chen, Yuanyuan Shi, Luoyi Fu, Weinan Zhang, Xinbing Wang, Chenghu Zhou, Zhouhan Lin, and Junxian He. 2023 · 2023
Later among the works it cites.
Hey Article, What Are You About? Question Answering for Information Systems Articles through Transformer Models for Long Sequences. In HICSS
Louisa Ebert, Sebastian Huettemann, and Roland M. Mueller. 2023 · 2023
Later among the works it cites.
Text Simplification of Scientific Texts for Non-Expert Readers. In SimpleText@CLEF-2023 . 2987–2998
Björn Engelmann, Fabian Haak, Christin Katharina Kreutz, Narjes Nikzad-Khasmakhi, and Philipp Schaer. 2023 · 2023
Later among the works it cites.
JaCoText: A Pretrained Model for Java Code-Text Generation
Jessica López Espejel, Mahaman Sanoussi Yahaya Alassan, Walid Dahhane, and El Hassane Ettifouri. 2023 · 2023
Later among the works it cites.
Bioformer: an efficient transformer language model for biomedical text mining
Li Fang, Qingyu Chen, Chih-Hsuan Wei, Zhiyong Lu, and Kai Wang. 2023 · 2023
Later among the works it cites.
Neural scaling of deep chemical models
Nathan C. Frey, Ryan Soklaski, Simon Axelrod, Siddharth Samsi, Rafael Gómez-Bombarelli, Connor W. Coley, and Vijay Gadepally. 2023 · 2023
Later among the works it cites.
Continual Pre-Training of Large Language Models: How to re-warm your model?. In ES-FoMo@ICML2023
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats Leon Richter, Quentin Gregory Anthony, Eugene Belilovsky, Irina Rish, and Timothée Lesort. 2023 · 2023
Later among the works it cites.
The Diminishing Returns of Masked Language Models to Science. In Findings of ACL . 1270–1283
Zhi Hong, Aswathy Ajith, James Pauloski, Eamon Duede, Kyle Chard, and Ian Foster. 2023 · 2023
Later among the works it cites.
FinBART: A Pre-trained Seq2seq Language Model for Chinese Financial Tasks. In Chinese National Conference on Computational Linguistics . 906–917
Dong Hongyuan, Che Wanxiang, He Xiaoyu, Zheng Guidong, and Wen Junjie. 2023 · 2023
Later among the works it cites.
Patton: Language Model Pretraining on Text-Rich Networks. In ACL . 7005–7020
Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han. 2023 · 2023
Later among the works it cites.
NLI4CT: Multi-Evidence Natural Language Inference for Clinical Trial Reports. In EMNLP . 16745–16764
Mael Jullien, Marco Valentino, Hannah Frost, Paul O’Regan, Dónal Landers, and Andre Freitas. 2023 · 2023
Later among the works it cites.
ManuBERT: A Pretrained Manufacturing Science Language Representation Model
Aman Kumar, Binil Starly, Collin Lynch, and Collin Lynch. 2023 · 2023
Later among the works it cites.
DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains. In ACL . 16207–16221
Yanis Labrak, Adrien Bazoge, Richard Dufour, Mickael Rouvier, Emmanuel Morin, Béatrice Daille, and Pierre-Antoine Gourraud. 2023 · 2023
Later among the works it cites.
Clinical-T5: Large Language Models Built Using MIMIC Clinical Text
Eric Lehman and Alistair Johnson. 2023 · 2023
Later among the works it cites.
Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining
Bingqian Lin, Zicong Chen, Mingjie Li, Haokun Lin, Hang Xu, Yi Zhu, Jianzhuang Liu, Wenjia Cai, Lei Yang, Shen Zhao, et al · 2023
Later among the works it cites.
Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023d · 2023
Later among the works it cites.
icsBERTs: Optimizing Pre-trained Language Models in Intelligent Customer Service. In INNS DLIA@IJCNN 2023 . 127–136
Shixuan Liu, Chen Peng, Chao Wang, Xiangyan Chen, and Shuangyong Song. 2023b · 2023
Later among the works it cites.
Leveraging molecular structure and bioactivity with chemical language models for de novo drug design
Michael Moret, Irene Pachon Angona, Leandro Cotos, Shen Yan, Kenneth Atz, Cyrill Brunner, Martin Baumgartner, Francesca Grisoni, and Gisbert Schneider. 2023 · 2023
Later among the works it cites.
COVID-Twitter-BERT: A natural language processing model to analyse COVID-19 content on Twitter
Martin Müller, Marcel Salathé, and Per E. Kummervold. 2023 · 2023
Later among the works it cites.
Can we Pretrain a SotA Legal Language Model on a Budget From Scratch?. In SustaiNLP . 158–182
Joel Niklaus and Daniele Giofre. 2023 · 2023
Later among the works it cites.
OpenAI. 2023 · 2023
Later among the works it cites.
BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition. In BioNLP . 337–349
Vera Pavlova and Mohammed Makhlouf. 2023 · 2023
Later among the works it cites.
Enriching Biomedical Knowledge for Low-resource Language Through Large-scale Translation. In EACL . 3131–3142
Long Phan, Tai Dang, Hieu Tran, Trieu H. Trinh, Vy Phan, Lam D. Chau, and Minh-Thang Luong. 2023 · 2023
Later among the works it cites.
Lightweight Transformers for Clinical Natural Language Processing
Omid Rohanian, Mohammadmahdi Nouriborji, Hannah Jauncey, Samaneh Kouchaki, ISARIC Clinical Characterisation Group, Lei Clifton, Laura Merson, and David A. Clifton. 2023 · 2023
Later among the works it cites.
MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education
Jia Tracy Shen, Michiharu Yamashita, Ethan Prihar, Neil Heffernan, Xintao Wu, Ben Graff, and Dongwon Lee. 2023 · 2023
Later among the works it cites.
A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing
Pranav Shetty, Arunkumar Chitteth Rajan, Chris Kuenneth, Sonakshi Gupta, Lakshmi Prerana Panchumarti, et al · 2023
Later among the works it cites.
Large language models encode clinical knowledge
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al · 2023
Later among the works it cites.
From Show to Tell: A Survey on Deep Learning-Based Image Captioning
Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Silvia Cascianelli, Giuseppe Fiameni, and Rita Cucchiara. 2023 · 2023
Later among the works it cites.
Fine-tuning large neural language models for biomedical natural language processing
Robert Tinn, Hao Cheng, Yu Gu, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2023 · 2023
Later among the works it cites.
CamemBERT-bio: a Tasty French Language Model Better for your Health
Rian Touchent, Laurent Romary, and Eric de la Clergerie. 2023 · 2023
Later among the works it cites.
Harnessing the Power of BERT in the Turkish Clinical Domain: Pretraining Approaches for Limited Data Scenarios. In Clinical Natural Language Processing Workshop . 161–170
Hazal Türkmen, Oguz Dikenelli, Cenk Eraslan, Mehmet Calli, and Suha Ozbek. 2023 · 2023
Later among the works it cites.
PharmBERT: a domain-specific BERT model for drug labels
Taha ValizadehAslani, Yiwen Shi, Ping Ren, Jing Wang, Yi Zhang, Meng Hu, Liang Zhao, and Hualou Liang. 2023 · 2023
Later among the works it cites.
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Xiao Wang, Guangyao Chen, Guangwu Qian, Pengcheng Gao, Xiao-Yong Wei, Yaowei Wang, Yonghong Tian, and Wen Gao. 2023a · 2023
Later among the works it cites.
RAMM: Retrieval-augmented Biomedical Visual Question Answering with Multi-modal Pre-training. In ACM MM . 547–556
Zheng Yuan, Qiao Jin, Chuanqi Tan, Zhengyun Zhao, Hongyi Yuan, Fei Huang, and Songfang Huang. 2023 · 2023
Later among the works it cites.
A Survey of Large Language Models
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al · 2023
Later among the works it cites.
Hongyin Zhu, Hao Peng, Zhiheng Lyu, Lei Hou, Juanzi Li, and Jinghui Xiao. 2023b · 2023
Later among the works it cites.
Unifying large language models and knowledge graphs: A roadmap
Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. 2024 · 2024
Closest in time.
Pre-trained Language Models in Biomedical Domain: A Systematic Survey
Benyou Wang, Qianqian Xie, Jiahuan Pei, Zhihong Chen, Prayag Tiwari, Zhao Li, and Jie Fu. 2024 · 2024
Closest in time.
Adversarial Examples for Evaluating Reading Comprehension Systems. In EMNLP . 2021–2031
Robin Jia and Percy Liang. 2017 · 2031
Closest in time.
Automated Chemical Reaction Extraction from Scientific Literature
Jiang Guo, A. Santiago Ibanez-Lopez, Hanyu Gao, Victor Quach, Connor W. Coley, Klavs F. Jensen, and Regina Barzilay. 2022 · 2045
Closest in time.