2020

Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning

Nie, Weili, Yu, Zhiding, Mao, Lei et al.

Understand

Humans have an inherent ability to learn novel concepts from only a few samples and generalize these concepts to different situations.

Even though today's machine learning models excel with a plethora of training data on standard recognition tasks, a considerable gap exists between machine-level pattern recognition and human-level concept learning.
To narrow this gap, the Bongard problems (BPs) were introduced as an inspirational challenge for visual cognition in intelligent systems.
Despite new advances in representation learning and learning to learn, BPs remain a daunting challenge for modern AI.

Built on

J. C. Raven, “Standardization of progressive matrices, 1938,” British Journal of Medical Psychology
1941
Earlier work this paper cites.
M. M. Bongard, “The recognition problem,” tech. rep., Foreign Technology Div Wright-Patterson AFB Ohio, 1968
1968
Earlier work this paper cites.
H. Abelson, N. Goodman, and L. Rudolph, “Logo manual,” 1974
1974
Earlier work this paper cites.
P. A. Carpenter, M. A. Just, and P. Shell, “What one intelligence test measures: a theoretical account of the processing in the raven progressive matrices test.,” Psychological review
1990
Earlier work this paper cites.
D. J. Chalmers, R. M. French, and D. R. Hofstadter, “High-level perception, representation, and analogy: A critique of artificial intelligence methodology,” Journal of Experimental & Theoretical Artificial Intelligence
1992
Earlier work this paper cites.
C. M. Bishop, “Mixture density networks,” 1994
1994
Earlier work this paper cites.
Basic books, 1995
D. R. Hofstadter, Fluid concepts and creative analogies: Computer models of the fundamental mechanisms of thought · 1995
Earlier work this paper cites.
K. Saito and R. Nakano, “A concept learning algorithm with adaptive search,” in Machine intelligence 14: applied machine intelligence
1996
Earlier work this paper cites.
A. Linhares, “A glimpse at the metaphysics of bongard problems,” Artificial Intelligence
2000
Earlier work this paper cites.
J. B. Tenenbaum and F. Xu, “Word learning as bayesian inference,” in Proceedings of the Annual Meeting of the Cognitive Science Society
2000
Earlier work this paper cites.
K. J. Holyoak, D. Gentner, and B. N. Kokinov, “Introduction: The place of analogy in cognition,” The analogical mind: Perspectives from cognitive science
2001
Earlier work this paper cites.
L. Fe-Fei et al
2003
Earlier work this paper cites.
H. Foundalis, “Phaeaco: A cognitive architecture inspired by bongard’s problems [ph. d. thesis],” Indiana University, Indiana, Bloomington
2006
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition
2009
Earlier work this paper cites.
E. Weitnauer and H. Ritter, “Physical bongard problems,” in Ifip international conference on artificial intelligence applications and innovations
2012
Earlier work this paper cites.
Springer Science & Business Media, 2013
B. Indurkhya, Metaphor and cognition: An interactionist approach · 2013
Earlier work this paper cites.
M. Kunda, K. McGreggor, and A. K. Goel, “A computational model for solving problems from the raven’s progressive matrices intelligence test using iconic visual representations,” Cognitive Systems Research
2013
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision
2014
Earlier work this paper cites.

Similar

S. K. Divvala, A. Farhadi, and C. Guestrin, “Learning everything about anything: Webly-supervised visual concept learning,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
2014
Cited alongside, same era.
B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,” Science
2015
Cited alongside, same era.
A. d. Garcez, T. R. Besold, L. De Raedt, P. Földiak, P. Hitzler, T. Icard, K.-U. Kühnberger, L. C. Lamb, R. Miikkulainen, and D. L. Silver, “Neural-symbolic learning and reasoning: contributions and challenges,” in 2015 AAAI Spring Symposium Series
2015
Cited alongside, same era.
G. Koch, R. Zemel, and R. Salakhutdinov, “Siamese neural networks for one-shot image recognition,” in ICML deep learning workshop
2015
Cited alongside, same era.
S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” in ICLR
2016
Cited alongside, same era.
A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap, “Meta-learning with memory-augmented neural networks,” in International conference on machine learning
2016
Cited alongside, same era.
O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al
2016
Cited alongside, same era.
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the 34th International Conference on Machine Learning-Volume 70
2017
Cited alongside, same era.
J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. Lawrence Zitnick, and R. Girshick, “Clevr: A diagnostic dataset for compositional language and elementary visual reasoning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
2017
Cited alongside, same era.
T. R. Besold, A. d. Garcez, S. Bader, H. Bowman, P. Domingos, P. Hitzler, K.-U. Kühnberger, L. C. Lamb, D. Lowd, P. M. V. Lima, et al
Original
2017
Cited alongside, same era.
J. Snell, K. Swersky, and R. S. Zemel, “Prototypical Networks for Few-shot Learning,” Advances in Neural Information Processing Systems
2017
Cited alongside, same era.
P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, et al
Original
2018
Cited alongside, same era.

Then

D. Ha and J. Schmidhuber, “World models,” arXiv preprint arXiv:1803.10122
Original
2018
Later among the works it cites.
E. Triantafillou, T. Zhu, V. Dumoulin, P. Lamblin, U. Evci, K. Xu, R. Goroshin, C. Gelada, K. Swersky, P.-A. Manzagol, et al
Original
2019
Later among the works it cites.
K. Lee, S. Maji, A. Ravichandran, and S. Soatto, “Meta-learning with differentiable convex optimization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
2019
Later among the works it cites.
M. Lázaro-Gredilla, D. Lin, J. S. Guntupalli, and D. George, “Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs,” Science Robotics
2019
Later among the works it cites.
J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu, “The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision,” in ICLR
2019
Later among the works it cites.
C. Han, J. Mao, C. Gan, J. Tenenbaum, and J. Wu, “Visual concept-metaconcept learning,” in Advances in Neural Information Processing Systems
2019
Later among the works it cites.
A. Bakhtin, L. van der Maaten, J. Johnson, L. Gustafson, and R. Girshick, “Phyre: A new benchmark for physical reasoning,” in Advances in Neural Information Processing Systems
2019
Later among the works it cites.
K. R. Allen, K. A. Smith, and J. B. Tenenbaum, “The tools challenge: Rapid trial-and-error learning in physical problem solving,” arXiv preprint arXiv:1907.09620
Original
2019
Later among the works it cites.
D. Saxton, E. Grefenstette, F. Hill, and P. Kohli, “Analysing mathematical reasoning abilities of neural models,” arXiv preprint arXiv:1904.01557
Original
2019
Later among the works it cites.
F. Chollet, “On the measure of intelligence,” arXiv preprint arXiv:1911.01547
Original
2019
Later among the works it cites.
C. Zhang, F. Gao, B. Jia, Y. Zhu, and S.-C. Zhu, “Raven: A dataset for relational and analogical visual reasoning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
2019
Later among the works it cites.
A. Raghu, M. Raghu, S. Bengio, and O. Vinyals, “Rapid learning or feature reuse? towards understanding the effectiveness of maml,” ICLR
2020
Closest in time.
Y. Chen, X. Wang, Z. Liu, H. Xu, and T. Darrell, “A new meta-baseline for few-shot learning,” arXiv preprint arXiv:2003.04390
Original
2020
Closest in time.
S. Kharagorgiev, “Solving bongard problems with deep learning,” k10v.github.io
2020
Closest in time.
Y. Tian, Y. Wang, D. Krishnan, J. B. Tenenbaum, and P. Isola, “Rethinking few-shot image classification: a good embedding is all you need?,” arXiv preprint arXiv:2003.11539
Original
2020
Closest in time.
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
2020
Closest in time.
D. Teney, P. Wang, J. Cao, L. Liu, C. Shen, and A. v. d. Hengel, “V-prom: A benchmark for visual reasoning using visual progressive matrices,” in AAAI
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…