A Continuous-time Perspective of Quasi-Newton Methods
发布时间:2019年12月23日
浏览次数:6501
发布者: Xiaoni Tan
主讲人: Honglin Yuan (Stanford University)
活动时间: 从 2019-12-30 10:00 到 11:00
场地: Room 9, Quan Zhai, BICMR
In this work, we study the continuous-time limit of quasi-Newton methods, one of the most widely used family of iterative algorithms for solving medium to large scale nonlinear equations g(x) = 0. The key idea is to utilize the secant conditions for identifying the correct scaling of the approximate Jacobian update equations. Focusing on the Broyden’s method, we study its connection with the continuous-time Newton’s method, and elucidate the caveat that prevents the (local or global) existence of its trajectories. We then propose some stabilization tricks motivated by momentum methods, and establish some preliminary theory on global convergence using a Lyapunov argument.