Developing a general-purpose clinical language inference model from a large corpus of clinical notes

Understand

Several biomedical language models have already been developed for clinical language inference.

• However, these models typically utilize general vocabularies and are trained on relatively small clinical corpora.
• We sought to evaluate the impact of using a domain-specific vocabulary and a large clinical training corpus on the performance of these language models in clinical language inference.
• We trained a Bidirectional Encoder Decoder from Transformers (BERT) model using a diverse, deidentified corpus of 75 million deidentified clinical notes authored at the University of California, San Francisco (UCSF).