Deep neural networks for physicists: from biological origins to neural operators
This seminar traces the history and mathematical foundations of deep learning for an audience of physicists. Starting from the formal neuron of McCulloch & Pitts (1943) and Hebb’s learning rule, we follow the evolution of Rosenblatt’s perceptron through to modern architectures, covering the AI winters and the renaissance brought about by gradient backpropagation. We progressively introduce the vector notation for dense layers, gradient computation via the chain rule expressed as a product of Jacobians, and the vanishing gradient phenomenon that long limited deep networks. The second part covers optimization algorithms (stochastic gradient descent, Adam), generalization guarantees, and key architectures — convolutional networks, residual networks, Transformers — that drove the current explosion in capabilities. The final part is dedicated to physics applications: learning PDE solutions through physics-informed neural networks (PINNs) and neural operators (FNO, Galerkin Transformer, Transolver), opening the way to differentiable simulators for fluid mechanics and computational physics.
Thursday 02 July at 10:15 in the CORIA Conference Room.