Stochastic variance reduced gradient (SVRG) algorithm was proposed by Johnson and Zhang in NeurIPS (2013) and has been extensively used in training neural networks. We shall rigorously prove that SVRG can be approximated by a family of stochastic differential delay equations ( SDDEs) under some conditions which include non-convex examples. It is well known that SDDEs have the effect of strong dissipations and variance reductions. Our result gives a new interpretation for SVRG. This is joint work with Peng Chen and Jianya Lu.

加入 Zoom 会议
https://us02web.zoom.us/j/89911480156?pwd=MFNCN29jdE1FYU84OXNlazBJZnhOUT09
Zoom ID：899 1148 0156
Password：086289