Applied Mathematical Seminar 44: Chebyshev Polynomial Acceleration for Stochastic Newton Methods

Seminar| Institute of Mathematical Sciences

Speaker: Jianyu Pan, School of Mathematical Sciences,  East China Normal University

Abstract: In this talk, we consider the stochastic Newton method for the large scale optimization problems arising from machine learning. In order to reduce the cost of computing Hessian and Hessian inverse, we propose to apply the Chebyshev polynomials to approximate the Hessian inverse. We show that, by utilizing the fast three-term recurrence formula, Chebyshev polynomial approximation can effectively reduce the computational cost. The convergence analysis are given and experiments on multiple benchmarks are carried out to illustrate the performance of our proposed algorithm.