Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing

Understand

BERT is the most recent Transformer-based model that achieves state-of-the-art performance in various NLP tasks.

• In this paper, we investigate the hardware acceleration of BERT on FPGA for edge computing.
• To tackle the issue of huge computational complexity and memory footprint, we propose to fully quantize the BERT (FQ-BERT), including weights, activations, softmax, layer normalization, and all the intermediate results.
• Experiments demonstrate that the FQ-BERT can achieve 7.94x compression for weights with negligible performance loss.