Understand
Recent work has shown that large pretrained Language Models (LMs) can not only perform remarkably well on a range of Natural Language Processing (NLP) tasks but also start improving on reasoning tasks such as arithmetic induction, symbolic manipulation, and commonsense reasoning with increasing size of models.
- However, it is still unclear what the underlying capabilities of these LMs are.
- Surprisingly, we find that these models have limitations on certain basic symbolic manipulation tasks such as copy, reverse, and addition.
- When the total number of symbols or repeating symbols increases, the model performance drops quickly.
Reading the bibliography…