ALBERT with Knowledge Graph Encoder Utilizing Semantic Similarity for Commonsense Question Answering

Understand

Recently, pre-trained language representation models such as bidirectional encoder representations from transformers (BERT) have been performing well in commonsense question answering (CSQA).

• However, there is a problem that the models do not directly use explicit information of knowledge sources existing outside.
• To augment this, additional methods such as knowledge-aware graph network (KagNet) and multi-hop graph relation network (MHGRN) have been proposed.
• In this study, we propose to use the latest pre-trained language model a lite bidirectional encoder representations from transformers (ALBERT) with knowledge graph information extraction technique.