Fetching the paper…

Reading the bibliography…

Does Invariant Risk Minimization Capture Invariance? · Around

2021

Does Invariant Risk Minimization Capture Invariance?

Kamath, Pritish, Tangella, Akilesh, Sutherland, Danica J. et al.

Original
PDF
Post on X

Understand

We show that the Invariant Risk Minimization (IRM) formulation of Arjovsky et al.

(2019) can fail to capture "natural" invariances, at least when used in its practical "linear" form, and even on very simple problems which directly follow the motivating examples for IRM.
This can lead to worse generalization on new environments, even when compared to unconstrained ERM.
The issue stems from a significant gap between the linear variant (as in their concrete method IRMv1) and the full non-linear IRM formulation.

Built on

“Invariant Risk Minimization”, 2019
Original
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani and David Lopez-Paz · 1907
Earlier work this paper cites.
“Distributionally Robust Optimization: A Review”, 2019
Original
Hamed Rahimian and Sanjay Mehrotra · 1908
Earlier work this paper cites.
“Invariant Risk Minimization Games”
Original
Kartik Ahuja, Karthikeyan Shanmugam, Kush. Varshney and Amit Dhurandhar · 2002
Earlier work this paper cites.
“Unshuffling Data for Improved Generalization”, 2020
Original
Damien Teney, Ehsan Abbasnejad and Anton van Hengel · 2002
Earlier work this paper cites.
“Invariant Rationalization”
Original
Shiyu Chang, Yang Zhang, Mo Yu and Tommi. Jaakkola · 2003
Earlier work this paper cites.
“A survey on domain adaptation theory: learning bounds and theoretical guarantees”, 2020
Original
Ievgen Redko, Emilie Morvant, Amaury Habrard, Marc Sebban and Younès Bennani · 2004
Earlier work this paper cites.

Similar

“In Search of Lost Domain Generalization”
Original
Ishaan Gulrajani and David Lopez-Paz · 2007
Cited alongside, same era.
“Empirical or Invariant Risk Minimization? A Sample Complexity Perspective”
Original
Kartik Ahuja, Jun Wang, Amit Dhurandhar, Karthikeyan Shanmugam and Kush. Varshney · 2010
Cited alongside, same era.
“MNIST handwritten digit database”
Yann LeCun, Corinna Cortes and CJ Burges · 2010
Cited alongside, same era.
“Understanding the failure modes of out-of-distribution generalization”
Original
Vaishnavh Nagarajan, Anders Andreassen and Behnam Neyshabur · 2010
Cited alongside, same era.
“The Risks of Invariant Risk Minimization”
Original
Elan Rosenfeld, Pradeep Ravikumar and Andrej Risteski · 2010

Then

“The Zero Set of a Real Analytic Function”, 2015
Original
Boris Mityagin · 2015
Later among the works it cites.
“Causal inference using invariant prediction: identification and confidence intervals”
Jonas Peters, Peter Bühlmann and Nicolai Meinshausen · 2015
Later among the works it cites.
“Invariant Causal Prediction for Nonlinear Models”
Christina Heinze-Deml, Jonas Peters and Nicolai Meinshausen · 2017
Later among the works it cites.
“Recognition in Terra Incognita”
Sara Beery, Grant Horn and Pietro Perona · 2018
Later among the works it cites.
“Invariant Models for Causal Transfer Learning”
Original
Mateo Rojas-Carulla, Bernhard Schölkopf, Richard Turner and Jonas Peters · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

“Differentiation under the Integral Sign”, 2013

Planet · 2013

Cited alongside, same era.