Backdrop: Stochastic Backpropagation

Understand

We introduce backdrop, a flexible and simple-to-implement method, intuitively described as dropout acting only along the backpropagation pipeline.

• Backdrop is implemented via one or more masking layers which are inserted at specific points along the network.
• Each backdrop masking layer acts as the identity in the forward pass, but randomly masks parts of the backward gradient propagation.
• Intuitively, inserting a backdrop layer after any convolutional layer leads to stochastic gradients corresponding to features of that scale.