AI-native Memory: A Pathway from LLMs Towards AGI

Understand

Large language models (LLMs) have demonstrated the world with the sparks of artificial general intelligence (AGI).

• One opinion, especially from some startups working on LLMs, argues that an LLM with nearly unlimited context length can realize AGI.
• However, they might be too optimistic about the long-context capability of (existing) LLMs -- (1) Recent literature has shown that their effective context length is significantly smaller than their claimed context length; and (2) Our reasoning-in-a-haystack experiments further demonstrate that simultaneously finding the relevant information from a long context and conducting (simple) reasoning is nearly impossible.
• In this paper, we envision a pathway from LLMs to AGI through the integration of \emph{memory}.