Gradient Boosted Decision Tree Neural Network

Understand

In this paper we propose a method to build a neural network that is similar to an ensemble of decision trees.

• We first illustrate how to convert a learned ensemble of decision trees to a single neural network with one hidden layer and an input transformation.
• We then relax some properties of this network such as thresholds and activation functions to train an approximately equivalent decision tree ensemble.
• The final model, Hammock, is surprisingly simple: a fully connected two layers neural network where the input is quantized and one-hot encoded.