PALT: Parameter-Lite Transfer of Language Models for Knowledge Graph Completion

Understand

This paper presents a parameter-lite transfer learning approach of pretrained language models (LM) for knowledge graph (KG) completion.

• Instead of finetuning, which modifies all LM parameters, we only tune a few new parameters while keeping the original LM parameters fixed.
• We establish this via reformulating KG completion as a "fill-in-the-blank" task, and introducing a parameter-lite encoder on top of the original LMs.
• We show that, by tuning far fewer parameters than finetuning, LMs transfer non-trivially to most tasks and reach competitiveness with prior state-of-the-art approaches.