Actor–Critic Methods for Optimal Control and Mean Field Games
发布时间:2026年10月26日
浏览次数:115
发布者: Congcong Zhao
主讲人: Prof. Mo ZhouMo Zhou(Peking University)
活动时间: 从 2026-11-04 10:00 到 11:30
场地: Room 9, Quan Zhai, BICMR
Abstract: Optimal control and mean field games provide mathematical frameworks for decision-making in dynamical systems, from controlling a single agent to understanding the collective behavior of large populations. I will begin with an overview of optimal control, including dynamic programming and Pontryagin's maximum principle, followed by a learning-based approach that brings ideas from reinforcement learning into continuous-time stochastic control. The approach uses an actor–critic framework in which the actor performs policy improvement through a policy-gradient flow, while the critic learns the value function using a variance-reduced temporal-difference formulation. Despite the nonconvex and coupled nature of the problem, the resulting single time-scale actor–critic flow achieves exponential convergence under suitable assumptions. The framework is then extended to mean field games by incorporating the evolution of the population distribution through an optimal-transport-based flow, leading to a unified mean field actor–critic method with convergence guarantees. Numerical examples illustrate the effectiveness of these methods for high-dimensional stochastic control and interacting-agent systems.
