Fetching the paper…

Reading the bibliography…

2019

Measuring Arithmetic Extrapolation Performance

Madsen, Andreas, Johansen, Alexander Rosenberg

Original
PDF
Post on X

Understand

The Neural Arithmetic Logic Unit (NALU) is a neural network layer that can learn exact arithmetic operations between the elements of a hidden state.

The goal of NALU is to learn perfect extrapolation, which requires learning the exact underlying logic of an unknown arithmetic problem.
Evaluating the performance of the NALU is non-trivial as one arithmetic problem might have many solutions.
As a consequence, single-instance MSE has been used to evaluate and compare performance between models.

Measuring Arithmetic Extrapolation Performance · Around

Built on

Probable inference, the law of succession, and statistical inference
Edwin B. Wilson · 1927
Earlier work this paper cites.
Adam: A Method for Stochastic Optimization
Original
Diederik P. Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Learning to execute
Original
Wojciech Zaremba and Ilya Sutskever · 2014
Earlier work this paper cites.

Similar

Neural gpus learn algorithms
Original
Lukasz Kaiser and Ilya Sutskever · 2016
Cited alongside, same era.
Grid long short-term memory
Original
Nal Kalchbrenner, Ivo Danihelka, and Alex Graves · 2016
Cited alongside, same era.
Improving the neural GPU architecture for algorithm learning
Original
Karlis Freivalds and Renars Liepins · 2017
Cited alongside, same era.

Then

Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden M. Lake and Marco Baroni · 2018
Later among the works it cites.
Neural arithmetic logic units
Andrew Trask, Felix Hill, Scott E Reed, Jack Rae, Chris Dyer, and Phil Blunsom · 2018
Later among the works it cites.
On evaluating the generalization of lstm models in formal languages
Mirac Suzgun, Yonatan Belinkov, and Stuart M. Shieber · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…