Fetching the paper…
Reading the bibliography…
Prior work has shown that text-conditioned diffusion models can learn to identify and manipulate primitive concepts underlying a compositional data-generating process, enabling generalization to entirely novel, out-of-distribution compositions.
The language of thought , volume 5
Jerry A Fodor et al · 1975
Earlier work this paper cites.
Tensor product variable binding and the representation of symbolic structures in connectionist systems
Paul Smolensky · 1990
Earlier work this paper cites.
Language, thought and compositionality
Jerry Fodor · 2001
Earlier work this paper cites.
Compositionality in rational analysis: Grammar-based induction for concept learning
Noah D Goodman, Joshua B Tenenbaum, Thomas L Griffiths, and Jacob Feldman · 2008
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale, 2021
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2010
Earlier work this paper cites.
Compositionality of rule representations in human prefrontal cortex
Carlo Reverberi, Kai Görgen, and John-Dylan Haynes · 2012
Earlier work this paper cites.
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
Andrew M Saxe, James L McClelland, and Surya Ganguli · 2013
Earlier work this paper cites.
Diversity techniques improve the performance of the best imbalance learning ensembles
José F Díez-Pastor, Juan J Rodríguez, César I García-Osorio, and Ludmila I Kuncheva · 2015
Earlier work this paper cites.
Deep residual learning for image recognition, 2015
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation, 2015
Olaf Ronneberger, Philipp Fischer, and Thomas Brox · 2015
Earlier work this paper cites.
Layer normalization, 2016
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton · 2016
Earlier work this paper cites.
Clevr: A diagnostic dataset for compositional language and elementary visual reasoning
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick · 2017
Earlier work this paper cites.
A convergence analysis of gradient descent for deep linear neural networks
Sanjeev Arora, Nadav Cohen, Noah Golowich, and Wei Hu · 2018
Earlier work this paper cites.
Compositional clustering in task structure learning
Nicholas T Franklin and Michael J Frank · 2018
Earlier work this paper cites.
Gradient descent aligns the layers of deep linear networks
Ziwei Ji and Matus Telgarsky · 2018
Earlier work this paper cites.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden Lake and Marco Baroni · 2018
Earlier work this paper cites.
An analytic theory of generalization dynamics and transfer learning in deep linear networks
Andrew K Lampinen and Surya Ganguli · 2018
Earlier work this paper cites.
Gradient descent quantizes relu network features
Hartmut Maennel, Olivier Bousquet, and Sylvain Gelly · 2018
Earlier work this paper cites.
Measuring compositionality in representation learning
Jacob Andreas · 2019
Earlier work this paper cites.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz · 2019
Earlier work this paper cites.
Implicit regularization in deep matrix factorization
Sanjeev Arora, Nadav Cohen, Wei Hu, and Yuping Luo · 2019
Earlier work this paper cites.
Width provably matters in optimization for deep linear neural networks
Simon Du and Wei Hu · 2019
Earlier work this paper cites.
Diversity in machine learning
Zhiqiang Gong, Ping Zhong, and Weidong Hu · 2019
Earlier work this paper cites.
Decoupled weight decay regularization, 2019
Ilya Loshchilov and Frank Hutter · 2019
Earlier work this paper cites.
A mathematical theory of semantic development in deep neural networks
Andrew M Saxe, James L McClelland, and Surya Ganguli · 2019
Earlier work this paper cites.
Training behavior of deep neural network in frequency domain
Zhi-Qin John Xu, Yaoyu Zhang, and Yanyang Xiao · 2019
Earlier work this paper cites.
High-dimensional dynamics of generalization error in neural networks
Madhu S Advani, Andrew M Saxe, and Haim Sompolinsky · 2020
Earlier work this paper cites.
Concepts and compositionality: in search of the brain’s language of thought
Steven M Frankland and Joshua D Greene · 2020
Earlier work this paper cites.
Early stopping in deep networks: Double descent and how to eliminate it
Reinhard Heckel and Fatih Furkan Yilmaz · 2020
Cited alongside, same era.
Compositionality decomposed: How do neural networks generalise?
Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni · 2020
Cited alongside, same era.
Zhiyuan Li, Yuping Luo, and Kaifeng Lyu · 2020
Cited alongside, same era.
Understanding the failure modes of out-of-distribution generalization
Vaishnavh Nagarajan, Anders Andreassen, and Behnam Neyshabur · 2020
Cited alongside, same era.
Prompting large pre-trained vision-language models for compositional concept learning
Guangyue Xu, Parisa Kordjamshidi, and Joyce Chai · 2022
Later among the works it cites.
When and why vision-language models behave like bag-of-words models, and what to do about it?
Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou · 2022
Later among the works it cites.
Do vision-language pretrained models learn primitive concepts?
Tian Yun, Usha Bhalla, Ellie Pavlick, and Chen Sun · 2022
Later among the works it cites.
A theory for emergence of complex skills in language models
Sanjeev Arora and Anirudh Goyal · 2023
Later among the works it cites.
A comprehensive benchmark of human-like relational reasoning for text-to-image foundation models
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski · 2020
Cited alongside, same era.
The role of syntactic planning in compositional image captioning
Emanuele Bugliarello and Desmond Elliott · 2021
Cited alongside, same era.
Unsupervised learning of compositional energy concepts
Yilun Du, Shuang Li, Yash Sharma, Josh Tenenbaum, and Igor Mordatch · 2021
Cited alongside, same era.
Variational diffusion models
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho · 2021
Cited alongside, same era.
Deep double descent: Where bigger models and more data hurt
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever · 2021
Cited alongside, same era.
Implicit bias of sgd for diagonal linear networks: a provable benefit of stochasticity
Scott Pesme, Loucas Pillaud-Vivien, and Nicolas Flammarion · 2021
Cited alongside, same era.
Zero-shot text-to-image generation, 2021
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever · 2021
Cited alongside, same era.
Visual representation learning does not generalize strongly within the same domain
Lukas Schott, Julius Von Kügelgen, Frederik Träuble, Peter Gehler, Chris Russell, Matthias Bethge, Bernhard Schölkopf, Francesco Locatello, and Wieland Brendel · 2021
Cited alongside, same era.
Colin Conwell and Tomer Ullman · 2023
Later among the works it cites.
Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl · 2023
Later among the works it cites.
(s) gd over diagonal linear networks: Implicit bias, large stepsizes and edge of stability
Mathieu Even, Scott Pesme, Suriya Gunasekar, and Nicolas Flammarion · 2023
Later among the works it cites.
Gaussian error linear units (gelus), 2023
Dan Hendrycks and Kevin Gimpel · 2023
Later among the works it cites.
Understanding incremental learning of gradient descent: A fine-grained analysis of matrix sensing
Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, and Jason D Lee · 2023
Later among the works it cites.
Ablating concepts in text-to-image diffusion models
Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, and Jun-Yan Zhu · 2023
Later among the works it cites.
Human-like systematic generalization through a meta-learning neural network
Brenden M Lake and Marco Baroni · 2023
Later among the works it cites.
Break it down: Evidence for structural compositionality in neural networks
Michael A Lepori, Thomas Serre, and Ellie Pavlick · 2023
Later among the works it cites.
Early neuron alignment in two-layer relu networks with small initialization
Hancheng Min, Enrique Mallada, and René Vidal · 2023
Later among the works it cites.
Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task
Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, and Hidenori Tanaka · 2023
Later among the works it cites.
Saddle-to-saddle dynamics in diagonal linear networks
Scott Pesme and Nicolas Flammarion · 2023
Later among the works it cites.
How capable can a transformer become? a study on synthetic, interpretable tasks
Rahul Ramesh, Mikail Khona, Robert P Dick, Hidenori Tanaka, and Ekdeep Singh Lubana · 2023
Later among the works it cites.
Rylan Schaeffer, Mikail Khona, Zachary Robertson, Akhilan Boopathy, Kateryna Pistunova, Jason W Rocks, Ila Rani Fiete, and Oluwasanmi Koyejo · 2023
Later among the works it cites.
What algorithms can transformers learn? a study in length generalization
Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, and Preetum Nakkiran · 2023
Later among the works it cites.
Compositional generative modeling: A single model is not all you need
Yilun Du and Leslie Kaelbling · 2024
Closest in time.
Instruct-skillmix: A powerful pipeline for llm instruction tuning
Simran Kaur, Simon Park, Anirudh Goyal, and Sanjeev Arora · 2024
Closest in time.
Towards an understanding of stepwise inference in transformers: A synthetic graph navigation model
Mikail Khona, Maya Okawa, Jan Hula, Rahul Ramesh, Kento Nishi, Robert Dick, Ekdeep Singh Lubana, and Hidenori Tanaka · 2024
Closest in time.
On the scalability of diffusion-based text-to-image generation
Hao Li, Yang Zou, Ying Wang, Orchid Majumder, Yusheng Xie, R Manmatha, Ashwin Swaminathan, Zhuowen Tu, Stefano Ermon, and Stefano Soatto · 2024
Closest in time.
A percolation model of emergence: Analyzing transformers trained on a formal language
Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P Dick, and Hidenori Tanaka · 2024
Closest in time.
Towards understanding epoch-wise double descent in two-layer linear neural networks
Amanda Olmin and Fredrik Lindsten · 2024
Closest in time.
Emergence of hidden capabilities: Exploring learning dynamics in concept space
Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep Singh Lubana, and Hidenori Tanaka · 2024
Closest in time.
Is human compositionality meta-learned?
Jacob Russin, Sam Whitman McGrath, Ellie Pavlick, and Michael J Frank · 2024
Closest in time.
Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip HS Torr, Adel Bibi, Samuel Albanie, and Matthias Bethge · 2024
Closest in time.
Compositional generalization from first principles
Thaddäus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel · 2024
Closest in time.