Sample Average Approximation for Black-Box Variational Inference
Sample Average Approximation for Black-Box Variational Inference
No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL
Han Wang, Archit Sakhadeo, Adam M White et al.
I wonder if anyone has written about #ASHA #hyperparameter optimization vs restart strategies.
I've been playing around with ASHA in #ray on my CMA-ES optimization task. Turns out my hyperparameters (population_size, sigma_0) are almost uncorrelated to performance, but the final performance varies a lot per run.
So what ASHA actually does for me is just picking the lucky runs. Which helped a lot. Maybe I should try this BIPOP restarting strategy of CMA-ES instead.