Speaker: Dr. Lin Xiao, Microsoft Research

Time: Jun. 25, 2015 14:00-15:00

Venue: Room 29 at Quan Zhai, BICMR

Abstract:
Stochastic gradient and coordinate descent methods are two classical methods in continuous optimization, each with a long and rich history. They are among the most efficient optimization algorithms for solving modern large-scale machine learning problems, mainly due to their very low computational cost per iteration (typically only process one or few examples or coordinates at a time). Their slow convergence rates usually are not a big concern for machine learning problems because of the training-generalization tradeoff.
I will present some recent progresses on new variants of stochastic gradient and coordinate descent methods for minimizing the average of a large number of simple convex functions. By exploiting the finite average structure, these algorithms achieve order of magnitude faster convergence rates than the classical methods, while maintaining the same low complexity per iteration. These advances significantly speed up machine learning practice with big data applications.