Improved Gradient-Based Optimization Over Discrete Distributions

Understand

In many applications we seek to maximize an expectation with respect to a distribution over discrete variables.

• Estimating gradients of such objectives with respect to the distribution parameters is a challenging problem.
• We analyze existing solutions including finite-difference (FD) estimators and continuous relaxation (CR) estimators in terms of bias and variance.
• We show that the commonly used Gumbel-Softmax estimator is biased and propose a simple method to reduce it.