Dynamical Low-Rank Approximation: from Radiation Therapy to Neural Networks

Abstract:
While neural networks, quantum mechanics, and radiation therapy appear very unrelated initially, they face the same shortcomings: high computational costs and a large memory footprint.  This stems from the fact that training weights in neural networks and evolving the wave function of quantum mechanics or radiation therapy dose in time require solving prohibitively large matrix or tensor ordinary differential equations.  It has, however, been observed that solutions in all three research fields exhibit low-rank structures.  This talk will discuss the use of dynamical low-rank approximation (DLRA), which has been introduced in to reduce costs and memory requirements in these application fields.  DLRA represents the solution as a low-rank matrix or tensor factorization and derives time evolution equations for the factors.  Solving these evolution equations requires the construction of robust and efficient time integrators.  The discussion focuses on the parallel integrator, which is inherently only first-order accurate.  Dr. Kusch will present a strategy to increase this integrator accuracy by extending the basis matrices and discuss how it can be used in tensor settings.  The talk concludes with numerical experiments for radiation therapy, quantum mechanics, and neural network training.

Speaker’s Bio:
Dr. Jonas Kusch is an Associate Professor in Scientific Computing at the Norwegian University of Life Sciences.  He received his Ph.D. in Mathematics from the Karlsruhe Institute of Technology in 2020.  Dr. Kusch’s research interests include numerical analysis for time integration, low-rank methods, and machine learning.  His work focuses on the construction of efficient numerical methods for radiation transport, quantum mechanics, and neural network training.