ZORB: A Derivative-Free Backpropagation Algorithm for Neural Networks

Understand

Gradient descent and backpropagation have enabled neural networks to achieve remarkable results in many real-world applications.

• Despite ongoing success, training a neural network with gradient descent can be a slow and strenuous affair.
• We present a simple yet faster training algorithm called Zeroth-Order Relaxed Backpropagation (ZORB).
• Instead of calculating gradients, ZORB uses the pseudoinverse of targets to backpropagate information.