Quantifying Inherent Randomness in Machine Learning Algorithms

Understand

Most machine learning (ML) algorithms have several stochastic elements, and their performances are affected by these sources of randomness.

• This paper uses an empirical study to systematically examine the effects of two sources: randomness in model training and randomness in the partitioning of a dataset into training and test subsets.
• We quantify and compare the magnitude of the variation in predictive performance for the following ML algorithms: Random Forests (RFs), Gradient Boosting Machines (GBMs), and Feedforward Neural Networks (FFNNs).
• Among the different algorithms, randomness in model training causes larger variation for FFNNs compared to tree-based methods.