Continuous-Time Accelerated Methods via a Hybrid Control Lens

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)

Abstract

Treating optimization methods as dynamical systems can be traced back centuries ago in order to comprehend the notions and behaviors of optimization methods. Lately, this mindset has become the driving force to design new optimization methods. Inspired by the recent dynamical system viewpoint of Nesterov's fast method, we propose two classes of fast methods, formulated as hybrid control systems, to obtain prespecified exponential convergence rate. Alternative to the existing fast methods, which are parametric-in-time second-order differential equations, we dynamically synthesize feedback controls in a state-dependent manner. Namely, in the first class, the damping term is viewed as the control input, while in the second class the amplitude with which the gradient of the objective function impacts the dynamics serves as the controller. The objective function requires to satisfy the so-called Polyak-Łojasiewicz inequality, which effectively implies no local optima and a certain gradient-domination property. Moreover, we establish that both hybrid structures possess Zeno-free solution trajectories. We finally provide a mechanism to determine the discretization step size to attain an exponential convergence rate.

Original languageEnglish
Article number8854991
Pages (from-to)3425-3440
JournalIEEE Transactions on Automatic Control
Volume65
Issue number8
DOIs
Publication statusPublished - 2020

Keywords

  • Dynamical systems
  • fast optimization methods
  • feedback control
  • hybrid control systems

Fingerprint Dive into the research topics of 'Continuous-Time Accelerated Methods via a Hybrid Control Lens'. Together they form a unique fingerprint.

  • Cite this