A blog by Robert M. Gower
Random Search
Notes on optimization, machine learning, and ideas worth trying. A little theory, a little experimentation.
A series of short posts
Stochastic optimization methods through statistical whitening
What can the statistics of a gradient tell us about how to take a step? Starting with variance, we explore a statistical viewpoint on modern optimizers.
Based on our paper with Tetiana Parshakova and Paulo Orenstein.