Fetching the paper…
Reading the bibliography…
Transformers, central to the successes in modern Natural Language Processing, often falter on arithmetic tasks despite their vast capabilities --which paradoxically include remarkable coding abilities.
Linear algebra with transformers
Charton, F. (2021) · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · 2021
Earlier work this paper cites.
The neural data router: Adaptive control flow in transformers improves systematic generalization
Csordás, R., Irie, K., and Schmidhuber, J. (2021) · 2021
Earlier work this paper cites.
Shape: Shifted absolute position embedding for transformers
Kiyono, S., Kobayashi, S., Suzuki, J., and Inui, K. (2021) · 2021
Earlier work this paper cites.
Investigating the limitations of transformers with simple arithmetic tasks
Nogueira, R., Jiang, Z., and Lin, J. (2021) · 2021
Earlier work this paper cites.
Train short, test long: Attention with linear biases enables input length extrapolation
Press, O., Smith, N. A., and Lewis, M. (2021) · 2021
Earlier work this paper cites.
Roformer: Enhanced transformer with rotary position embedding
Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B., and Liu, Y. (2021) · 2021
Earlier work this paper cites.
What is my math transformer doing?–three results on interpretability and generalization
Charton, F. (2022) · 2022
Earlier work this paper cites.
Systematic generalization and emergent structures in transformers trained on structured tasks
Li, Y. and McClelland, J. L. (2022) · 2022
Earlier work this paper cites.
Limitations of language models in arithmetic and symbolic induction
Qian, J., Wang, H., Li, Z., Li, S., and Yan, X. (2022) · 2022
Cited alongside, same era.
Solving math word problems with process-and outcome-based feedback
Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., and Higgins, I. (2022) · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022) · 2022
Cited alongside, same era.
Unveiling transformers with lego: a synthetic reasoning task
Zhang, Y., Backurs, A., Bubeck, S., Eldan, R., Gunasekar, S., and Wagner, T. (2022) · 2022
Cited alongside, same era.
Length generalization in arithmetic transformers
Jelassi, S., d’Ascoli, S., Domingo-Enrich, C., Wu, Y., Li, Y., and Charton, F. (2023) · 2023
Closest in time.
The impact of positional encoding on length generalization in transformers
Kazemnejad, A., Padhi, I., Ramamurthy, K. N., Das, P., and Reddy, S. (2023) · 2023
Closest in time.
Teaching arithmetic to small transformers
Lee, N., Sreenivasan, K., Lee, J. D., Lee, K., and Papailiopoulos, D. (2023) · 2023
Closest in time.
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. (2023) · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Zhou, H., Nova, A., Larochelle, H., Courville, A., Neyshabur, B., and Sedghi, H. (2022) · 2022
Cited alongside, same era.
Sparks of artificial general intelligence: Early experiments with gpt-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · 2023
Cited alongside, same era.
Faith and fate: Limits of transformers on compositionality
Dziri, N., Lu, X., Sclar, M., Li, X. L., Jian, L., Lin, B. Y., West, P., Bhagavatula, C., Bras, R. L., Hwang, J. D., et al. (2023) · 2023
Cited alongside, same era.
Hanna, M., Liu, O., and Variengien, A. (2023) · 2023
Cited alongside, same era.
OpenAI (2023) · 2023
Closest in time.
Randomized positional encodings boost length generalization of transformers
Ruoss, A., Delétang, G., Genewein, T., Grau-Moya, J., Csordás, R., Bennani, M., Legg, S., and Veness, J. (2023) · 2023
Closest in time.
Testolin, A. (2023) · 2023
Closest in time.
Gpt can solve mathematical problems without a calculator
Yang, Z., Ding, M., Lv, Q., Jiang, Z., He, Z., Guo, Y., Bai, J., and Tang, J. (2023) · 2023
Closest in time.