Skip to yearly menu bar Skip to main content


Poster

Generative Flow Networks as Entropy-Regularized RL

Daniil Tiapkin · Nikita Morozov · Alexey Naumov · Dmitry Vetrov

MR1 & MR2 - Number 170
[ ]
Sat 4 May 6 a.m. PDT — 8:30 a.m. PDT
 
Oral presentation: Oral: Probabilistic Methods
Thu 2 May 6:45 a.m. PDT — 8 a.m. PDT

Abstract:

The recently proposed generative flow networks (GFlowNets) are a method of training a policy to sample compositional discrete objects with probabilities proportional to a given reward via a sequence of actions. GFlowNets exploit the sequential nature of the problem, drawing parallels with reinforcement learning (RL). Our work extends the connection between RL and GFlowNets to a general case. We demonstrate how the task of learning a generative flow network can be efficiently redefined as an entropy-regularized RL problem with a specific reward and regularizer structure. Furthermore, we illustrate the practical efficiency of this reformulation by applying standard soft RL algorithms to GFlowNet training across several probabilistic modeling tasks. Contrary to previously reported results, we show that entropic RL approaches can be competitive against established GFlowNet training methods. This perspective opens a direct path for integrating reinforcement learning principles into the realm of generative flow networks.

Live content is unavailable. Log in and register to view live content