This paper introduces a general patch-based model-free framework for despeckling of single and multi-baseline synthetic aperture radar (SAR) image. Particularly, the method is based on the empirical distribution similarity between the patch containing the pixel to be restored and the patch containing a candidate similar pixel. In order to decide whether the patches follow a similar distribution, the Kolmogorov-Smirnov test is adapted. Finally, within the restoration process, the selected similar pixels are aggregated based on their relative importance obtained according to their distribution similarities. Experimental validation of the proposed methodology is provided using different real data sets and compared with existing NLSAR approach in relation to single SAR image despeckling and tomographic application for the 3D reflectivity reconstruction of volumetric media as well as permanent scatterer detection in urban environments.