A Continuous-time Perspective of Quasi-Newton Methods
Time: 2019-12-23
Published By: Xiaoni Tan
Speaker(s): Honglin Yuan (Stanford University)
Time: 10:00-11:00 December 30, 2019
Venue: Room 9, Quan Zhai, BICMR
In this work, we study the continuous-time limit of quasi-Newton methods, one of the most widely used family of iterative algorithms for solving medium to large scale nonlinear equations g(x) = 0. The key idea is to utilize the secant conditions for identifying the correct scaling of the approximate Jacobian update equations. Focusing on the Broyden’s method, we study its connection with the continuous-time Newton’s method, and elucidate the caveat that prevents the (local or global) existence of its trajectories. We then propose some stabilization tricks motivated by momentum methods, and establish some preliminary theory on global convergence using a Lyapunov argument.