Learning with symmetries: eigenvectors, graph representations and sample complexity
In many applications, especially in the sciences, data and tasks have known invariances. Encoding such invariances directly into a machine learning model can improve learning outcomes, while it also poses challenges on efficient model design. In the first part of the talk, we will focus on the invariances relevant to eigenvectors and eigenspaces being inputs to a neural network. Such inputs are important, for instance, for graph representation learning, point clouds and graphics. We will discuss targeted architectures that express the relevant invariances or equivariances - sign flips and changes of basis - and their theoretical and empirical benefits in different applications. Second, we will take a broader, theoretical perspective. Empirically, it is known that encoding invariances into the machine learning model can reduce sample complexity. What can we say theoretically? We will look at example results for various settings and models.
This talk is based on joint work with Derek Lim, Joshua Robinson, Behrooz Tahmasebi, Thien Le, Hannah Lawrence, Bobak Kiani, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron and Melanie Weber.
