WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain

Understand

Pre-trained language models have shown impressive performance on a variety of tasks and domains.

• Previous research on financial language models usually employs a generic training scheme to train standard model architectures, without completely leveraging the richness of the financial data.
• We propose a novel domain specific Financial LANGuage model (FLANG) which uses financial keywords and phrases for better masking, together with span boundary objective and in-filing objective.
• Additionally, the evaluation benchmarks in the field have been limited.