Trajectory balance: Improved credit assignment in GFlowNets

Understand

Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or strings, from a given unnormalized density by sequences of actions, where many possible action sequences may lead to the same object.

• We find previously proposed learning objectives for GFlowNets, flow matching and detailed balance, which are analogous to temporal difference learning, to be prone to inefficient credit propagation across long action sequences.
• We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives.
• We prove that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution.