Zero-shot Entity Linking with Efficient Long Range Sequence Modeling

Understand

This paper considers the problem of zero-shot entity linking, in which a link in the test time may not present in training.

• Following the prevailing BERT-based research efforts, we find a simple yet effective way is to expand the long-range sequence modeling.
• Unlike many previous methods, our method does not require expensive pre-training of BERT with long position embedding.
• Instead, we propose an efficient position embeddings initialization method called Embedding-repeat, which initializes larger position embeddings based on BERT-Base.