Fetching the paper…
Reading the bibliography…
As large language models (LLMs) become increasingly advanced, their ability to exhibit compositional generalization -- the capacity to combine learned skills in novel ways not encountered during training -- has garnered significant attention.
Tree-structured composition in neural networks without tree-structured architectures
Samuel R Bowman, Christopher D Manning, and Christopher Potts · 2015
Earlier work this paper cites.
Diagnostic classifiers revealing how neural networks process hierarchical structure
Sara Veldhoen, Dieuwke Hupkes, Willem H Zuidema, et al · 2016
Earlier work this paper cites.
Scan: Learning hierarchical compositional visual concepts
Irina Higgins, Nicolas Sonnerat, Loic Matthey, Arka Pal, Christopher P Burgess, Matko Bosnjak, Murray Shanahan, Matthew Botvinick, Demis Hassabis, and Alexander Lerchner · 2017
Earlier work this paper cites.
Improving text-to-sql evaluation methodology
Catherine Finegan-Dollak, Jonathan K Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev · 2018
Earlier work this paper cites.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden Lake and Marco Baroni · 2018
Earlier work this paper cites.
Structured disentangled representations
Babak Esmaeili, Hao Wu, Sarthak Jain, Alican Bozkurt, Narayanaswamy Siddharth, Brooks Paige, Dana H Brooks, Jennifer Dy, and Jan-Willem Meent · 2019
Earlier work this paper cites.
Mathijs Mul and Willem Zuidema · 2019
Earlier work this paper cites.
Analysing mathematical reasoning abilities of neural models
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli · 2019
Earlier work this paper cites.
Compositionality and generalization in emergent languages
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni · 2020
Earlier work this paper cites.
Compositional visual generation and inference with energy based models, 2020
Yilun Du, Shuang Li, and Igor Mordatch · 2020
Earlier work this paper cites.
Compositionality decomposed: how do neural networks generalise?, 2020
Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni · 2020
Earlier work this paper cites.
Measuring compositional generalization: A comprehensive method on realistic data, 2020
Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, Dmitry Tsarkov, Xiao Wang, Marc van Zee, and Olivier Bousquet · 2020
Cited alongside, same era.
Compositional generalization by learning analytical expressions
Qian Liu, Shengnan An, Jian-Guang Lou, Bei Chen, Zeqi Lin, Yan Gao, Bin Zhou, Nanning Zheng, and Dongmei Zhang · 2020
Cited alongside, same era.
On the dangers of stochastic parrots: Can language models be too big?
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell · 2021
Cited alongside, same era.
Learning to compose visual relations, 2021
Nan Liu, Shuang Li, Yilun Du, Joshua B. Tenenbaum, and Antonio Torralba · 2021
Cited alongside, same era.
Training compute-optimal large language models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al · 2022
Tinystories: How small can language models be and still speak coherent english?, 2023
Ronen Eldan and Yuanzhi Li · 2023
Later among the works it cites.
T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation, 2023
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu · 2023
Later among the works it cites.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Later among the works it cites.
Measuring and narrowing the compositionality gap in language models, 2023
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Making transformers solve compositional tasks, 2022
Santiago Ontañón, Joshua Ainslie, Vaclav Cvicek, and Zachary Fisher · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Cited alongside, same era.
Emergent abilities of large language models, 2022
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus · 2022
Cited alongside, same era.
Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language
Zhenlin Xu, Marc Niethammer, and Colin A Raffel · 2022
Cited alongside, same era.
A theory for emergence of complex skills in language models
Sanjeev Arora and Anirudh Goyal · 2023
Cited alongside, same era.
Skills-in-context prompting: Unlocking compositionality in large language models
Jiaao Chen, Xiaoman Pan, Dian Yu, Kaiqiang Song, Xiaoyang Wang, Dong Yu, and Jianshu Chen · 2023
Cited alongside, same era.
Later among the works it cites.
Collie: Systematic construction of constrained text generation tasks, 2023
Shunyu Yao, Howard Chen, Austin W. Hanjie, Runzhe Yang, and Karthik Narasimhan · 2023
Later among the works it cites.
Skill-mix: a flexible and expandable family of evaluations for ai models
Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal, and Sanjeev Arora · 2023
Later among the works it cites.
Compositional generative modeling: A single model is not all you need, 2024
Yilun Du and Leslie Kaelbling · 2024
Closest in time.
Faith and fate: Limits of transformers on compositionality
Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, et al · 2024
Closest in time.
Evaluating text-to-visual generation with image-to-text generation, 2024
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan · 2024
Closest in time.
Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task, 2024
Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, and Hidenori Tanaka · 2024
Closest in time.