Mokhwa Lee, Ph.D. Research Proficiency Presentation: 'Multi-point Secant Quasi-Newton Methods'

Tuesday, November 23, 2021 - 11:00am to 12:30pm
Zoom, contact for Zoom info.
Event Description: 

Abstract: When dealing with a large scale optimization problem, classical second-order methods, such as Newton’s method, are no longer practical because it requires iteratively solving a large scale linear system of order n. For this reason, Quasi-Newton (QN) methods, like BFGS [1–4] or Broyden’s method [5], are introduced because they are more efficient than Newton’s method. An example of one such popular QN method in machine learning is L-BFGS [6]. This project focuses on multisecant extensions of QN methods. These methods compose a Hessian estimate via rank-1 and rank-2 updates involving variable and gradient successive differences. By satisfying multiple multisecant conditions (our extension), we hope to improve the Hessian inverse estimate of traditional QN methods like Broyden’s method and BFGS. In our initial study, we explored several ways to maintain the Hessian inverse estimate, accepting and rejecting older updates by thresholding several nondegeneracy metrics: nonzero determinants, small inner products, large smallest eigenvalue, and small condition number. Our preliminary work focuses on improving the modification of Broyden’s method, but our goal is to extend these techniques to broader QN methods such as BFGS and their limited memory versions.

Computed Event Type: