Adaptive Stochastic Optimization: A Framework for Analyzing Stochastic Optimization Algorithms

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Adaptive Stochastic Optimization: A Framework for Analyzing Stochastic Optimization Algorithms

By: 
Frank E. Curtis; Katya Scheinberg

Optimization lies at the heart of machine learning (ML) and signal processing (SP). Contemporary approaches based on the stochastic gradient (SG) method are nonadaptive in the sense that their implementation employs prescribed parameter values that need to be tuned for each application. This article summarizes recent research and motivates future work on adaptive stochastic optimization methods, which have the potential to offer significant computational savings when training largescale systems.

Optimization lies at the heart of machine learning (ML) and signal processing (SP). Contemporary approaches based on the stochastic gradient (SG) method are nonadaptive in the sense that their implementation employs prescribed parameter values that need to be tuned for each application. This article summarizes recent research and motivates future work on adaptive stochastic optimization methods, which have the potential to offer significant computational savings when training large-scale systems.

The success of stochastic optimization algorithms for solving problems arising in ML and SP are now widely recognized. Scores of articles have appeared in recent years as researchers aim to build on fundamental methodologies, such as the SG method [26]. The motivation and scope of many of these efforts have been captured in various books and review articles;

SPS on Twitter

  • This Wednesday, join the Information Forensics and Security Technical Committee Webinar Series when Dr. Richard Heu… https://t.co/ORdtuq5SlQ
  • Our Biomedical Imaging and Signal Processing Webinar Series continues on Tuesday, 5 July when Michael Unser present… https://t.co/7bYh8ZPHI0
  • Join us TODAY at 11:00 AM ET when the Brain Space Initiative Talk Series continues with Dr. Tianming Liu presenting… https://t.co/MEfnzk6dAE
  • Our 75th anniversary is approaching in 2023, and we're celebrating with a Special Issue of IEEE Signal Processing M… https://t.co/U6UNv8kLSO
  • The SPS Webinar Series continues on Monday, 20 June when Dr. Zhijin Qin presents "Semantic Communications: Principl… https://t.co/FhI7aP3GLi

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar