Skip to yearly menu bar Skip to main content


Coarse-Grained Smoothness for Reinforcement Learning in Metric Spaces

Omer Gottesman · Kavosh Asadi · Cameron Allen · Samuel Lobel · George Konidaris · Michael Littman

Auditorium 1 Foyer 47


Principled decision-making in continuous state--action spaces is impossible without some assumptions. A common approach is to assume Lipschitz continuity of the Q-function. We show that, unfortunately, this property fails to hold in many typical domains. We propose a new coarse-grained smoothness definition that generalizes the notion of Lipschitz continuity, is more widely applicable, and allows us to compute significantly tighter bounds on Q-functions, leading to improved learning. We provide a theoretical analysis of our new smoothness definition, and discuss its implications and impact on control and exploration in continuous domains.

Live content is unavailable. Log in and register to view live content