Fetching the paper…

Reading the bibliography…

What to Learn, and How: Toward Effective Learning from Rationales · Around

2021

What to Learn, and How: Toward Effective Learning from Rationales

Carton, Samuel, Kanoria, Surya, Tan, Chenhao

Original
PDF
Post on X

Understand

Learning from rationales seeks to augment model prediction accuracy using human-annotated rationales (i.e.

subsets of input tokens) that justify their chosen labels, often in the form of intermediate or multitask supervision.
While intuitive, this idea has proven elusive in practice.
We make two observations about human rationales via empirical analyses: 1) maximizing rationale supervision accuracy is not necessarily the optimal objective for improving model accuracy; 2) human rationales vary in whether they provide sufficient information for the model to exploit for prediction.

Built on

Regularizing Black-box Models for Improved Interpretability
Original
Gregory Plumb, Maruan Al-Shedivat, Angel Alexander Cabrera, Adam Perer, Eric Xing, and Ameet Talwalkar. 2020 · 1902
Earlier work this paper cites.
Interpretable Neural Predictions with Differentiable Binary Variables
Original
Jasmijn Bastings, Wilker Aziz, and Ivan Titov. 2020 · 1905
Earlier work this paper cites.
HuggingFace’s Transformers: State-of-the-art Natural Language Processing
Original
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2020 · 1910
Earlier work this paper cites.
Rethinking Cooperative Rationalization: Introspective Extraction and Complement Control
Original
Mo Yu, Shiyu Chang, Yang Zhang, and Tommi S. Jaakkola. 2019 · 1910
Earlier work this paper cites.
ERASER: A Benchmark to Evaluate Rationalized NLP Models
Original
Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. 2019 · 1911
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J. Williams. 1992 · 1992
Earlier work this paper cites.
Language Models are Few-Shot Learners
Original
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2005
Earlier work this paper cites.
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Original
Bhargavi Paranjape, Mandar Joshi, John Thickstun, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020 · 2005
Earlier work this paper cites.
Aligning Faithful Interpretations with their Social Attribution
Original
Alon Jacovi and Yoav Goldberg. 2021 · 2006
Earlier work this paper cites.
Explainable Artificial Intelligence: a Systematic Review
Original
Giulia Vilone and Luca Longo. 2020 · 2006
Earlier work this paper cites.
Robustness to Spurious Correlations via Human Annotations
Original
Megha Srivastava, Tatsunori Hashimoto, and Percy Liang. 2020 · 2007
Earlier work this paper cites.
Using “Annotator Rationales” to Improve Machine Learning for Text Categorization
Omar Zaidan, Jason Eisner, and Christine Piatko. 2007 · 2007
Earlier work this paper cites.
Active Learning by Labeling Features
Gregory Druck, Burr Settles, and Andrew McCallum. 2009 · 2009
Earlier work this paper cites.

Similar

Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
Original
Peter Hase, Shiyue Zhang, Harry Xie, and Mohit Bansal. 2020 · 2010
Cited alongside, same era.
LIREx: Augmenting Language Inference with Relevant Explanation
Original
Xinyan Zhao and V. G. Vinod Vydiswaran. 2020 · 2012
Cited alongside, same era.
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015 · 2015
Cited alongside, same era.
Rationalizing Neural Predictions
Original
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016 · 2016
Cited alongside, same era.
Categorical Reparameterization with Gumbel-Softmax
Original

Then

Inferring Which Medical Treatments Work from Reports of Clinical Trials
Eric Lehman, Jay DeYoung, Regina Barzilay, and Byron C. Wallace. 2019 · 2019
Later among the works it cites.
Evaluating and Characterizing Human Rationales
Samuel Carton, Anirudh Rathore, and Chenhao Tan. 2020 · 2020
Later among the works it cites.
Invariant Rationalization
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. 2020 · 2020
Later among the works it cites.
Interpretations are Useful: Penalizing Explanations to Align Neural Networks with Prior Knowledge
Laura Rieger, Chandan Singh, William Murdoch, and Bin Yu. 2020 · 2020
Later among the works it cites.
Rationalization through Concepts
Original
Diego Antognini and Boi Faltings. 2021 · 2021
Closest in time.
When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Eric Jang, Shixiang Gu, and Ben Poole. 2017 · 2017

Cited alongside, same era.

Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. 2017 · 2017

Cited alongside, same era.

e-snli: Natural language inference with natural language explanations

Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018 · 2018

Cited alongside, same era.

Extractive Adversarial Networks: High-Recall Explanations for Identifying Personal Attacks in Social Media Posts

Samuel Carton, Qiaozhu Mei, and Paul Resnick. 2018 · 2018

Cited alongside, same era.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Cited alongside, same era.

Training classifiers with natural language explanations

Braden Hancock, Paroma Varma, Stephanie Wang, Martin Bringmann, Percy Liang, and Christopher Ré. 2018 · 2018

Cited alongside, same era.

Looking beyond the surface: A challenge set for reading comprehension over multiple sentences

Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018 · 2018

Cited alongside, same era.

FEVER: a large-scale dataset for fact extraction and verification

James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018 · 2018

Cited alongside, same era.

Peter Hase and Mohit Bansal. 2021 · 2021

Closest in time.

Human Rationales as Attribution Priors for Explainable Stance Detection

Sahil Jayaram and Emily Allaway. 2021 · 2021

Closest in time.

Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretations

Neil Jethani, Mukund Sudarshan, Yindalon Aphinyanaphongs, and Rajesh Ranganath. 2021 · 2021

Closest in time.

Explanation-Based Human Debugging of NLP Models: A Survey

Piyawat Lertvittayakumjorn and Francesca Toni. 2021 · 2021

Closest in time.

Reframing Human-AI Collaboration for Generating Free-Text Explanations

Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2021 · 2021

Closest in time.

Teach Me to Explain: A Review of Datasets for Explainable NLP

Sarah Wiegreffe and Ana Marasović. 2021 · 2021

Closest in time.

Refining Neural Networks with Compositional Explanations

Original

Huihan Yao, Ying Chen, Qinyuan Ye, Xisen Jin, and Xiang Ren. 2021 · 2021

Closest in time.

Understanding Interlocking Dynamics of Cooperative Rationalization

Mo Yu, Yang Zhang, Shiyu Chang, and Tommi S. Jaakkola. 2021 · 2021

Closest in time.