Align to Structure: Aligning Large Language Models with Structural Information

Understand

Generating long, coherent text remains a challenge for large language models (LLMs), as they lack hierarchical planning and structured organization in discourse generation.

• We introduce Structural Alignment, a novel method that aligns LLMs with human-like discourse structures to enhance long-form text generation.
• By integrating linguistically grounded discourse frameworks into reinforcement learning, our approach guides models to produce coherent and well-organized outputs.
• We employ a dense reward scheme within a Proximal Policy Optimization framework, assigning fine-grained, token-level rewards based on the discourse distinctiveness relative to human writing.