David Pfau @davidpfau.com · May 2 Most RL algorithms for continuous actions fall into two classes. PPO, MPO, etc... work for arbitrary policies but only use scalar value information. DDPG, SAC, TD3, etc… use gradients of the value but are limited to policies that can be represented by reparameterization.
6 likes 1 replies
?
Replies
Previous work using Wasserstein gradient flows for continuous control arrived at methods similar to SAC which use the reparameterization trick, but we found that there is a simple and general closed form update that works for *any* stochastic policy.