Large Language Models with Controllable Working Memory

Understand

Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), owing to their excellent understanding and generation abilities.

• Remarkably, what further sets these models apart is the massive amounts of world knowledge they internalize during pretraining.
• While many downstream applications provide the model with an informational context to aid its performance on the underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored.
• As a desirable behavior, an LLM should give precedence to the context whenever it contains task-relevant information that conflicts with the model's memorized knowledge.