Raman-based sensors represent a promising solution to enable both detection and fingerprinting of anionic pollutants in the water distribution network. Due to the weak nature of Raman scattering, a signal intensity enhancement mechanism, such as surface enhanced Raman spectroscopy (SERS), is required. Given the combination of SERS being a first layer effect and the low affinity for metallic surfaces shown by anions, functionalization of the SERS substrates using positively charged self-assembled monolayers (SAMs) is required to guarantee a strong SERS signal. In this work, the performance of three thiol-based coatings, namely, 2-mercapto-4-methyl-pyrimidine, cysteamine, and 2-dimethyl-amino-ethanethiol, is systematically compared for the detection of nitrite, nitrate, and perchlorate ions in water. For each coating, the limit of detection of those analytes is studied in combination with commercial SERS substrates. Cysteamine-coated SERS substrates are shown to provide the lowest limit of detection for the three analytes of this study. Evaluation of this coating on real drinking water samples is reported.