Afficher la couverture 'SGD with Coordinate Sampling: Theory and Practice | Leluc, Rémi' en grand format
/5

0 avis

SGD with Coordinate Sampling: Theory and Practice

Archive ouverte

Leluc, Rémi | Portier, François

Edité par CCSD ; Microtome Publishing -

Journal of Machine Learning Research 2022. International audience. While classical forms of stochastic gradient descent algorithm treat the different coordinates in the same way, a framework allowing for adaptive (non uniform) coordinate sampling is developed to leverage structure in data. In a non-convex setting and including zeroth order gradient estimate, almost sure convergence as well as non-asymptotic bounds are established. Within the proposed framework, we develop an algorithm, MUSKETEER, based on a reinforcement strategy: after collecting information on the noisy gradients, it samples the most promising coordinate (all for one); then it moves along the one direction yielding an important decrease of the objective (one for all). Numerical experiments on both synthetic and real data examples confirm the effectiveness of MUSKETEER in large scale problems.

Suggestions

Du même auteur

Sliced-Wasserstein Estimation with Spherical Harmonics as Control Variates

Archive ouverte | Leluc, Rémi | CCSD

The Sliced-Wasserstein (SW) distance between probability measures is defined as the average of the Wasserstein distances resulting for the associated one-dimensional projections. As a consequence, the SW distance can be written as...

A Quadrature Rule combining Control Variates and Adaptive Importance Sampling

Archive ouverte | Leluc, Rémi | CCSD

International audience. Driven by several successful applications such as in stochastic gradient descent or in Bayesian computation, control variates have become a major tool for Monte Carlo integration. However, st...

Feature Clustering for Support Identification in Extreme Regions

Archive ouverte | Jalalzai, Hamid | CCSD

International audience. Understanding the complex structure of multivariate extremes is a major challenge in various fields from portfolio monitoring and environmental risk management to insurance. In the framework ...

Chargement des enrichissements...