Gaspard Lambrechts

@gsprd.be

2151 Followers 657 Following

Postdoctoral researcher working on RL in POMDP at McGill and Mila - gsprd.be

ICML poster of the paper « A Theoretical Justification for Asymmetric Actor-Critic Algorithms » by Gaspard Lambrechts, Damien Ernst and Aditya Mahajan.
Cover page of the PhD thesis "Reinforcement Learning in Partially Observable Markov Decision Processes: Learning to Remember the Past by Learning to Predict the Future" by Gaspard Lambrechts
Theorem showing the finite-time suboptimality bound for the asymmetric and symmetric actor-critic algorithms. The asymmetric algorithm has four terms: the natural actor-critic term, the gradient estimation term, the residual gradient term, and the average critic error. The symmetric algorithm has an additional term: the inference term.
Theorem showing the finite-time error bound for the asymmetric and symmetric temporal difference learning algorithms. The asymmetric algorithm has three terms: the temporal difference learning term, the function approximation term, and the bootstrapping shift term. The symmetric algorithm has an additional term: the aliasing term.
Title page of the paper "A Theoretical Justification for Asymmetric Actor-Critic Algorithms", written by Gaspard Lambrechts, Damien Ernst and Aditya Mahajan.
Figure showing the policy being passed the feature z = f(h) of the history, and the critic being passed both the feature z = f(h) of the history and the state s as input. The asymmetric critic and the policy are used together to form the sample policy-gradient expression.
Figure showing the history h being compressed into a feature z = f(h) for then being passed to a policy g(a | z).
Table showing that the MDP assumes full state observability both during training and execution and that the POMDP assumes partial state observability both during training and execution, while the privileged POMDP assumes full state observability during training but partial state observability during execution.
Slide showing three recent successes of reinforcement learning that have used an asymmetric actor-critic algorithm: - Magnetic Control of Tokamak Plasma through Deep RL (Degrave et al., 2022). - Champion-Level Drone Racing using Deep RL (Kaufmann et al., 2023). - A Super-Human Vision-Based RL Agent in Gran Turismo (Vasco et al., 2024).
Typst interface showing an example of Slydst code and the resulting slides.