2018

Deep Neural Decision Trees

Yang, Yongxin, Morillo, Irene Garcia, Hospedales, Timothy M.

Understand

Deep neural networks have been proven powerful at processing perceptual data, such as images and audio.

However for tabular data, tree-based models are more popular.
A nice property of tree-based models is their natural interpretability.
In this work, we present Deep Neural Decision Trees (DNDT) -- tree models realised by neural networks.

Built on

Classification and Regression Trees
Breiman, L., H. Friedman, J., A. Olshen, R., and J. Stone, C · 1984
Earlier work this paper cites.
C4.5: Programs for Machine Learning
Quinlan, J. Ross · 1993
Earlier work this paper cites.
Supervised and unsupervised discretization of continuous features
Dougherty, James, Kohavi, Ron, and Sahami, Mehran · 1995
Earlier work this paper cites.
The lack of a priori distinctions between learning algorithms
Wolpert, David H · 1996
Earlier work this paper cites.
The random subspace method for constructing decision forests
Ho, Tin Kam · 1998
Earlier work this paper cites.
Random forests
Breiman, Leo · 2001
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E · 2011
Earlier work this paper cites.
Estimating or propagating gradients through stochastic neurons
Original
Bengio, Yoshua · 2013
Earlier work this paper cites.
The ethics of artificial intelligence , pp. 316–334
Bostrom, Nick and Yudkowsky, Eliezer · 2014
Earlier work this paper cites.
Neural decision forests for semantic image labelling
Bulò, S. and Kontschieder, P · 2014
Earlier work this paper cites.

Similar

TensorFlow: Large-scale machine learning on heterogeneous systems, 2015
Abadi, Martín, Agarwal, Ashish, Barham, Paul, Brevdo, Eugene, Chen, Zhifeng, Citro, Craig, Corrado, Greg S., Davis, Andy, Dean, Jeffrey, Devin, Matthieu, Ghemawat, Sanjay, Goodfellow, Ian, Harp, Andrew, Irving, Geoffrey, Isard, Michael, Jia, Yangqing, Jozefowicz, Rafal, Kaiser, Lukasz, Kudlur, Manjunath, Levenberg, Josh, Mané, Dandelion, Monga, Rajat, Moore, Sherry, Murray, Derek, Olah, Chris, Schuster, Mike, Shlens, Jonathon, Steiner, Benoit, Sutskever, Ilya, Talwar, Kunal, Tucker, Paul, Vanhoucke, Vincent, Vasudevan, Vijay, Viégas, Fernanda, Vinyals, Oriol, Warden, Pete, Wattenberg, Martin, Wicke, Martin, Yu, Yuan, and Zheng, Xiaoqiang · 2015
Cited alongside, same era.
Learning interpretable classification rules using sequential rowsampling
Dash, S., Malioutov, D. M., and Varshney, K. R · 2015
Cited alongside, same era.
Deep neural decision forests
Kontschieder, P., Fiterau, M., Criminisi, A., and Bulò, S. R · 2015
Cited alongside, same era.
Deep learning
Lecun, Yann, Bengio, Yoshua, and Hinton, Geoffrey · 2015
Cited alongside, same era.
Efficient non-greedy optimization of decision trees
Norouzi, Mohammad, Collins, Maxwell D., Johnson, Matthew, Fleet, David J., and Kohli, Pushmeet · 2015
Cited alongside, same era.
Deep learning in neural networks: An overview
Schmidhuber, J · 2015
Cited alongside, same era.
Xgboost: A scalable tree boosting system
Chen, Tianqi and Guestrin, Carlos · 2016
Cited alongside, same era.
Examples are not enough, learn to criticize! Criticism for interpretability
Kim, Been, Khanna, Rajiv, and Koyejo, Sanmi · 2016
Cited alongside, same era.
”why should i trust you?”: Explaining the predictions of any classifier
Ribeiro, Marco Tulio, Singh, Sameer, and Guestrin, Carlos · 2016
Cited alongside, same era.

Then

Neural Decision Trees
Balestriero, R · 2017
Later among the works it cites.
Hierarchical Multiscale Recurrent Neural Networks
Chung, J., Ahn, S., and Bengio, Y · 2017
Later among the works it cites.
Towards a rigorous science of interpretable machine learning
Doshi-Velez, Finale; Kim, Been · 2017
Later among the works it cites.
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., and Poole, B · 2017
Later among the works it cites.
TCAV: Relative concept importance testing with Linear Concept Activation Vectors
Kim, B., Gilmer, J., Viegas, F., Erlingsson, U., and Wattenberg, M · 2017
Later among the works it cites.
Learning interpretable classification rules with boolean compressed sensing
Malioutov, Dmitry M., Varshney, Kush R., Emad, Amin, and Dash, Sanjeeb · 2017
Later among the works it cites.
Automatic differentiation in pytorch
Paszke, Adam, Gross, Sam, Chintala, Soumith, Chanan, Gregory, Yang, Edward, DeVito, Zachary, Lin, Zeming, Desmaison, Alban, Antiga, Luca, and Lerer, Adam · 2017
Later among the works it cites.
Challenges for transparency
Weller, Adrian · 2017
Later among the works it cites.
Learning decision trees with reinforcement learning
Xiong, Zheng, Zhang, Wenpeng, and Zhu, Wenwu · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…