Improving Systematic Generalization Through Modularity and Augmentation

Understand

Systematic generalization is the ability to combine known parts into novel meaning; an important aspect of efficient human learning, but a weakness of neural network learning.

• In this work, we investigate how two well-known modeling principles -- modularity and data augmentation -- affect systematic generalization of neural networks in grounded language learning.
• We analyze how large the vocabulary needs to be to achieve systematic generalization and how similar the augmented data needs to be to the problem at hand.
• Our findings show that even in the controlled setting of a synthetic benchmark, achieving systematic generalization remains very difficult.