Improving the Neural GPU Architecture for Algorithm Learning

Understand

Algorithm learning is a core problem in artificial intelligence with significant implications on automation level that can be achieved by machines.

• Recently deep learning methods are emerging for synthesizing an algorithm from its input-output examples, the most successful being the Neural GPU, capable of learning multiplication.
• We present several improvements to the Neural GPU that substantially reduces training time and improves generalization.
• We introduce a new technique - hard nonlinearities with saturation costs- that has general applicability.