Assessing Phrase Break of ESL Speech with Pre-trained Language Models and Large Language Models

Understand

This work introduces approaches to assessing phrase breaks in ESL learners' speech using pre-trained language models (PLMs) and large language models (LLMs).

• There are two tasks: overall assessment of phrase break for a speech clip and fine-grained assessment of every possible phrase break position.
• To leverage NLP models, speech input is first force-aligned with texts, and then pre-processed into a token sequence, including words and phrase break information.
• To utilize PLMs, we propose a pre-training and fine-tuning pipeline with the processed tokens.