635. Adaptive restart of conservative dynamics for convex optimization
Invited abstract in session MD-32: Algorithms for machine learning and inverse problems: adaptive strategies, stream Advances in large scale nonlinear optimization.
Monday, 14:30-16:00Room: 41 (building: 303A)
Authors (first author is the speaker)
| 1. | Alessandro Scagliotti
|
| CIT School, TU Munich, and MCML |
Abstract
In the last years, tools from Dynamical Systems have been fruitfully applied to study existing accelerated optimization methods and to develop new ones. Typically, continuous-time models for accelerated methods are mechanical systems with damping.
In this talk, we present an optimization method based on a conservative mechanical system, where the objective function plays the role of the potential energy.
Due to the absence of damping, the convergence of this method completely relies on the adaptive restart strategy: a) the initial velocity is set equal to 0; b) by the conservation of the mechanical energy, part of the initial potential energy is transformed into kinetic energy; c) when a proper restart condition is met, the velocity is reset to zero and the kinetic energy at the restart time is instantly dissipated.
We prove the convergence result both for the continuous-time method and for the discrete-time version. Finally, we discuss some possible extensions to the nonsmooth case, with particular focus on l1-regularization.
[1] A. Scagliotti, P. Colli Franzone. A piecewise conservative method for unconstrained convex optimization, COAP (2022).
[2] A. Scagliotti, P. Colli Franzone. A subgradient method with constant step-size for l1-composite optimization, BUMI (2023).
Keywords
- Convex Optimization
- Algorithms
Status: accepted
Back to the list of papers