Uncovering Hidden Training Dynamics in Neural Networks via Inter-Sample Influence Graphs

Deep learning models are primarily trained through batchwise optimization, where each update can potentially be a tug‑of‑war among samples, shaping the overall trajectory of learning. Existing interpretability tools, most notably influence functions, have provided valuable insights into how individual training samples affect model predictions, primarily at test time. However, these methods were not intended to capture these inter-sample interactions that arise during training. Here, we ask a complementary question: How does optimizing the loss on one training sample affect the loss on the rest during learning? We introduce Influence Graphs (IGs), directed inter-sample graphs where each edge weight $w_{ij}$ quantifies how optimizing on sample $X_i$ influences the loss of sample $X_j$. We estimate these influences via simulated batch interventions and slope coefficients of loss changes, enabling scalable construction of IGs during training. We further define the {Mean-of-Mean In-Degree Influence (MMDI)} and prove it bounds generalization under practical assumptions. Empirically, MMDI correlates strongly with test accuracy in noisy-label settings, making it a useful diagnostic of model quality even before test metrics are available. Finally, we show that IGs reveal distinct, evolving training phases, offering a new lens on the dynamics of learning.