ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding

Understand

Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding.

• However, most existing methods lack the systematic mining and utilization of layout-centered knowledge, leading to sub-optimal performances.
• In this paper, we propose ERNIE-Layout, a novel document pre-training solution with layout knowledge enhancement in the whole workflow, to learn better representations that combine the features from text, layout, and image.
• Specifically, we first rearrange input sequences in the serialization stage, and then present a correlative pre-training task, reading order prediction, to learn the proper reading order of documents.