Segmenting Natural Language Sentences via Lexical Unit Analysis

Understand

In this work, we present Lexical Unit Analysis (LUA), a framework for general sequence segmentation tasks.

• Given a natural language sentence, LUA scores all the valid segmentation candidates and utilizes dynamic programming (DP) to extract the maximum scoring one.
• LUA enjoys a number of appealing properties such as inherently guaranteeing the predicted segmentation to be valid and facilitating globally optimal training and inference.
• Besides, the practical time complexity of LUA can be reduced to linear time, which is very efficient.