Initial Guessing Bias: How Untrained Networks Favor Some Classes

Understand

Understanding and controlling biasing effects in neural networks is crucial for ensuring accurate and fair model performance.

• In the context of classification problems, we provide a theoretical analysis demonstrating that the structure of a deep neural network (DNN) can condition the model to assign all predictions to the same class, even before the beginning of training, and in the absence of explicit biases.
• We prove that, besides dataset properties, the presence of this phenomenon, which we call \textit{Initial Guessing Bias} (IGB), is influenced by model choices including dataset preprocessing methods, and architectural decisions, such as activation functions, max-pooling layers, and network depth.
• Our analysis of IGB provides information for architecture selection and model initialization.