Skip to yearly menu bar Skip to main content


Implicit Regularization in Deep Tucker Factorization: Low-Rankness via Structured Sparsity

Kais HARIZ · Hachem Kadri · Stephane Ayache · Maher Moakher · Thierry Artieres

MR1 & MR2 - Number 156
[ ]
Fri 3 May 8 a.m. PDT — 8:30 a.m. PDT


We theoretically analyze the implicit regularization of deep learning for tensor completion. We show that deep Tucker factorization trained by gradient descent induces a structured sparse regularization. This leads to a characterization of the effect of the depth of the neural network on the implicit regularization and provides a potential explanation for the bias of gradient descent towards solutions with low multilinear rank. Numerical experiments confirm our theoretical findings and give insights into the behavior of gradient descent in deep tensor factorization.

Live content is unavailable. Log in and register to view live content