Kriging aims to estimate the attributes of unseen geo-locations from observations in the spatial vicinity or physical connections.Existing works assume that neighbors' information offers the basis for estimating the unobserved target while ignoring non-neighbors.However, neighbors could also be quite different or even misleading, and the non-neighbors could still offer constructive information.To this end, we propose "Contrastive-Prototypical" self-supervised learning for Kriging (KCP): (1) The neighboring contrastive module coarsely pushes neighbors together and non-neighbors apart. (2) In parallel, the prototypical module identifies similar representations via exchanged prediction, such that it refines the misleading neighbors and recycles the useful non-neighbors from the neighboring contrast component. As a result, not all the neighbors and some of the non-neighbors will be used to infer the target. (3) To learn general and robust representations, we design an adaptive augmentation module that encourages data diversity. Theoretical bound is derived for the proposed augmentation. Extensive experiments on real-world datasets demonstrate the superior performance of KCP compared to its peers with 6\% improvements and exceptional transferability and robustness.