Seminario Machine Learning
Diffusion Flows and Optimal Transport in Machine Learning
Ponente: Gabriel Peyre (École Normale Supérieure (Paris))Fecha: viernes 21 de marzo de 2025 - 12:00Lugar: Aula Naranja, ICMAT
Online: https://us02web.zoom.us/j/82628856695?pwd=Z2otBrFqDz2Aoe5GnKQbFRcqRDmoy4.1
Resumen:
In this talk, I will review how concepts from optimal transport can be applied to analyze seemingly unrelated machine learning methods for sampling and training neural networks. The focus is on using optimal transport to study dynamical flows in the space of probability distributions. The first example will be sampling by flow matching, which regresses advection fields. In its simplest case (diffusion models), this approach exhibits a gradient structure similar to the displacement seen in optimal transport. I will then discuss Wasserstein gradient flows, where the flow minimizes a functional within the optimal transport geometry. This framework can be employed to model and understand the training dynamics of the probability distribution of neurons in two-layer networks. The final example will explore modeling the evolution of the probability distribution of tokens in deep transformers. This requires modifying the optimal transport structure to accommodate the softmax normalization inherent in attention mechanisms.
