Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction

Understand

Deep learning (DL) based predictive models from electronic health records (EHR) deliver impressive performance in many clinical tasks.

• Large training cohorts, however, are often required to achieve high accuracy, hindering the adoption of DL-based models in scenarios with limited training data size.
• Recently, bidirectional encoder representations from transformers (BERT) and related models have achieved tremendous successes in the natural language processing domain.
• The pre-training of BERT on a very large training corpus generates contextualized embeddings that can boost the performance of models trained on smaller datasets.