The Nonlinearity Coefficient - A Practical Guide to Neural Architecture Design

Understand

In essence, a neural network is an arbitrary differentiable, parametrized function.

• Choosing a neural network architecture for any task is as complex as searching the space of those functions.
• For the last few years, 'neural architecture design' has been largely synonymous with 'neural architecture search' (NAS), i.e.
• brute-force, large-scale search.