Skip to yearly menu bar Skip to main content


Poster

Maximum entropy GFlowNets with soft Q-learning

Sobhan Mohammadpour · Emmanuel Bengio · Emma Frejinger · Pierre-Luc Bacon

MR1 & MR2 - Number 7
[ ]
Fri 3 May 8 a.m. PDT — 8:30 a.m. PDT

Abstract:

Generative Flow Networks (GFNs) have emerged as a powerful tool for sampling discrete objects from unnormalized distributions, offering a scalable alternative to Markov Chain Monte Carlo (MCMC) methods. While GFNs draw inspiration from maximum entropy reinforcement learning (RL), the connection between the two has largely been unclear and seemingly applicable only in specific cases. This paper addresses the connection by constructing an appropriate reward function, thereby establishing an exact relationship between GFNs and maximum entropy RL. This construction allows us to introduce maximum entropy GFNs, which achieve the maximum entropy attainable by GFNs without constraints on the state space, in contrast to GFNs with uniform backward policy.

Live content is unavailable. Log in and register to view live content