BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching

Understand

This paper presents a deep neural architecture, for Natural Language Sentence Matching (NLSM) by adding a deep recursive encoder to BERT so called BERT with Deep Recursive Encoder (BERT-DRE).

• Our analysis of model behavior shows that BERT still does not capture the full complexity of text, so a deep recursive encoder is applied on top of BERT.
• Three Bi-LSTM layers with residual connection are used to design a recursive encoder and an attention module is used on top of this encoder.
• To obtain the final vector, a pooling layer consisting of average and maximum pooling is used.