Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land

Understand

Neural networks surround us, in the form of large language models, speech transcription systems, molecular discovery algorithms, robotics, and much more.

• Stripped of anything else, neural networks are compositions of differentiable primitives, and studying them means learning how to program and how to interact with these models, a particular example of what is called differentiable programming.
• This primer is an introduction to this fascinating field imagined for someone, like Alice, who has just ventured into this strange differentiable wonderland.
• I overview the basics of optimizing a function via automatic differentiation, and a selection of the most common designs for handling sequences, graphs, texts, and audios.