Flat Multi-modal Interaction Transformer for Named Entity Recognition

Understand

Multi-modal named entity recognition (MNER) aims at identifying entity spans and recognizing their categories in social media posts with the aid of images.

• However, in dominant MNER approaches, the interaction of different modalities is usually carried out through the alternation of self-attention and cross-attention or over-reliance on the gating machine, which results in imprecise and biased correspondence between fine-grained semantic units of text and image.
• To address this issue, we propose a Flat Multi-modal Interaction Transformer (FMIT) for MNER.
• Specifically, we first utilize noun phrases in sentences and general domain words to obtain visual cues.