Actor–Critic Methods for Optimal Control and Mean Field Games
Time: 2026-10-26
Published By: Congcong Zhao
Speaker(s): Prof. Mo ZhouMo Zhou(Peking University)
Time: 10:00-11:30 November 4, 2026
Venue: Room 9, Quan Zhai, BICMR
Abstract: Optimal control and mean field games provide mathematical frameworks for decision-making in dynamical systems, from controlling a single agent to understanding the collective behavior of large populations. I will begin with an overview of optimal control, including dynamic programming and Pontryagin's maximum principle, followed by a learning-based approach that brings ideas from reinforcement learning into continuous-time stochastic control. The approach uses an actor–critic framework in which the actor performs policy improvement through a policy-gradient flow, while the critic learns the value function using a variance-reduced temporal-difference formulation. Despite the nonconvex and coupled nature of the problem, the resulting single time-scale actor–critic flow achieves exponential convergence under suitable assumptions. The framework is then extended to mean field games by incorporating the evolution of the population distribution through an optimal-transport-based flow, leading to a unified mean field actor–critic method with convergence guarantees. Numerical examples illustrate the effectiveness of these methods for high-dimensional stochastic control and interacting-agent systems.
