Mete Soner
Professor of Operations Research and Financial Engineering
Princeton University
Title: Synchronization Games
Abstract: Building on Winfree's work, the Kuramoto model (1975) has become the cornerstone of mathematical models of collective synchronization, and has received attention in all natural sciences, engineering, and mathematics. These autonomous oscillators are coupled through a nonlinear interaction term which plays a central role in the long term behavior of the system. While the classical model postulates the dynamics of each oscillator in the form of a system of nonlinear ordinary differential equations, Yin, Mehta, Meyn, & Shanbhag (2010) use the mean-field game (MFG) formalism of Lasry & Lions, and Huang, Caines. & Malhame. In this talk, in addition to the Yin et.al model, we also introduce a simpler two state model which can be seen as a discretization of the original one. We outline results showing that the mean field approach also delivers the same type of results including the phase transition from incoherence to synchronization. In particular, in the discrete setting we provide a comprehensive characterization of stationary and dynamic equilibria along with their stability properties. In all models, while the system is unsynchronized when the coupling is not sufficiently strong, fascinatingly, they exhibit an abrupt transition to a full synchronization above a critical value of the interaction parameter. In the subcritical regime, the uniform distribution representing incoherence is the only stationary equilibrium. Above the critical interaction threshold, the uniform equilibrium becomes unstable and there is a multiplicity of stationary equilibria that are self-organizing. The discrete model with discounted cost present dynamic equilibria that spiral around the uniform distribution before converging to the self-organizing equilibria. With an ergodic cost, however, unexpected periodic equilibria around the uniform distribution emerge, which also minimize the associated mean-field control problem.
This talk is based on joint works with Rene Carmona and Felix Hoefer of Princeton, and Quentin Cormier of INRIA.
Wenpin Tang
Assistant Professor
Columbia IEOR
Title: Contractive diffusion models and score matching by continuous reinforcement learning.
Abstract: In this talk, I will link two different topics. The past decade has witnessed the success of generative modeling (e.g. GANS, VAEs,...) in creating high quality samples in a wide variety of data modalities. The first part of this talk is concerned with the recently developed diffusion models, the key idea of which is to reverse a certain stochastic dynamics. I will first take a continuous-time perspective, and examine the performance of different SDE schemes including VE (variance exploding) and VP (variance preserving). Our idea is to "contract" the reversed dynamics leading to possible new diffusion model designs.
In the second part, I will talk about continuous reinforcement learning. Reinforcement Learning (RL) has been successfully applied to wide-ranging domains in the past decade. Recent years have witnessed a fast growing body of research that has extended the frontiers of continuous RL such as designing model-free methods and algorithms. I will discuss the recently introduced "q-learning" and closely related policy optimization. Finally, I will highlight a natural application of continuous RL to fine-tune the score function in the diffusion models. Other aspects such as reinforcement learning with human feedback (RLHF) and direct preference optimization (DPO) will be discussed if time permits.
