Skip to main content

Education Center

A Deep Dive into Recent Advances in Stochastic Approximation (Slides)

SHARE:
Category
Proficiency
Language
Media Type
Pricing

SPS Members $0.00
IEEE Members $11.00
Non-members $15.00

Authors
Date

This mini-tutorial provides an introduction to the recent development of the stochastic approximation (SA) scheme. The first part introduces the essential foundation of the SA scheme as a general device for locating the roots of mean field functions. [1]

The presenter then showcases how the SA scheme can be applied to examples in signal processing and machine learning, such as compressed training, online expectation maximization, reinforcement learning, performative prediction, etc., that go beyond the use of gradient updates. This demonstrates the versatility of SA in analyzing and understanding classical algorithms. [1, 2, 3]

The second part introduces a general theory for the convergence of the SA scheme, accommodating scenarios with non-gradient updates and biased oracles. He will emphasize modern convergence analyses that encompass sample complexity and high-probability bounds. [1]

With these latest innovations, participants will be well-equipped to explore new frontiers in applying SA in their research. This is an abridged version of a tutorial given at ICASSP 2025, which was jointly given with Eric Moulines (Ecole Polytechnique) and Gersende Fort (CNRS), and the materials are based upon the overview paper "Stochastic Approximation beyond Gradient for Signal Processing and Machine Learning," published in IEEE TSP in 2023. [1, 2]

Duration
Slides: 50
Subtitles
✖

IEEE SPS Education Center FAQs

The IEEE SPS Education Center is your hub for educational resources in signal processing. It offers a variety of materials tailored for students and professionals alike. You can explore content based on your specific interests and skill levels.

Select the program and click on the external link to the IEEE SPS Resource Center.

Educational credits in the form of professional development hours (PDHs) or continuing education units (CEUs) are available on select educational programs.