PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model

Understand

In this work we focus on fine-tuning a pre-trained BERT model and applying it to patent classification.

• When applied to large datasets of over two millions patents, our approach outperforms the state of the art by an approach using CNN with word embeddings.
• In addition, we focus on patent claims without other parts in patent documents.
• Our contributions include: (1) a new state-of-the-art method based on pre-trained BERT model and fine-tuning for patent classification, (2) a large dataset USPTO-3M at the CPC subclass level with SQL statements that can be used by future researchers, (3) showing that patent claims alone are sufficient for classification task, in contrast to conventional wisdom.