Uncovering Hidden Training Dynamics in Neural Networks via Inter-Sample Influence Graphs

Influence functions have been widely used to analyze model predictions, primarily at test time, by estimating how individual training samples affect specific outcomes. While valuable for interpretability, outlier detection, and dataset pruning, these methods treat samples in isolation and overlook how they interact during training. In this work, we ask a complementary question: \emph{How does optimizing the loss on one training sample affect the loss on others during learning?} We introduce {Influence Graphs (IGs)}, directed inter-sample graphs where each edge weight $w_{ij}$ quantifies how optimizing on sample $X_i$ influences the loss of sample $X_j$. We estimate these influences via simulated batch interventions and slope coefficients of loss changes, enabling scalable construction of IGs during training. We further define the {Mean-of-Mean In-Degree Influence (MMDI)} and prove it bounds generalization under practical assumptions. Empirically, MMDI correlates strongly with test accuracy in noisy-label settings, making it a useful diagnostic of model quality even before test metrics are available. Finally, we show that IGs reveal distinct, evolving training phases, offering a new lens on the dynamics of learning.