Backtracking Line Search and Applications
Professor Tuyen Truong
, University of Oslo
Abstract:
(Armijo’s) Backtracking line search is a simple but efficient method to boost convergence of iterative optimization methods. This talk will give first an overview of optimization problems, what are minimal requirements of a good method to solve them, then recall about Backtracking line search (and some other popular methods) to boost convergence. Then, applications of Backtracking line search are described for:
- Some variants of gradient descent method, with usage in training Deep Neural networks
- Some variants of Newton’s method, with usage in finding roots of a meromorphic function
- DC (difference of convex functions) programming
- Stochastic optimization
The talk is self-contained and to follow most of the talk it is only required a good knowledge of multiple-variable calculus.
Speaker’s Bio:
Dr. Tuyen Trung Truong, a professor at the University of Oslo, works in both pure and applied mathematics, including optimization and applications to/of deep learning. In particular, he has designed several new first and second order iterative methods with better global convergence guarantees for optimization and finding roots of systems of equations. Some of these methods have been tested with deep neural networks also. He also has worked with using deep learning for healthcare. He received a Ph.D. in mathematics from Indiana University, Bloomington in 2012, had postdoc positions at Syracuse University, the Korea Institute for Advanced Study (KIAS) in South Korea and the University of Adelaide in Australia. Since 2017, he has been working at the Department of Mathematics, University of Oslo, first as an associate professor, and from 2023 as a professor. He is also a cofounder of some tech startups.
