Electrical excitation of surface plasmons via metal-insulator-metal tunneling junctions (M-I-M TJs) has recently been demonstrated to have experimental conversion efficiencies of 1% [W. Du et al. Nat. Photonics 11, 623 (2017)], 5–6 orders of magnitude higher than theoretically predicted by Parzefall and Novotny [ACS Photonics 5, 4195 (2018)]. In this work we resolve this discrepancy between theory and experiment, and report a rigorous analytical and experimental study on the role of surface roughness in the near-field coupling of the initially excited M-I-M TJ cavity surface plasmon polariton (M-I-M SPP) to the daughter radiative and nonradiative modes. We find that varying the roughness profile of the M-I-M TJ significantly (which is determined with atomic force microscopy and cross-section scanning transmission electron microscopy) improves the near-field outcoupling efficiency for two reasons: the effective thickness of the electrodes reduces with increasing roughness, and roughness provides momentum matching for mode overlap. The role of surface roughness in near-field outcoupling is analysed quantitatively by incorporating a conformal random roughness profile in the finite-element electromagnetic modeling of a plasmonically active M-I-M TJ. We show that the outcoupling efficiency can be enhanced up to 15% for the M-I-M SPP coupling to daughter modes, and demonstrate an overall electron-to-plasmon conversion efficiency of 1.5% (based on an inelastic tunneling efficiently of 10%), supporting recent experimental findings. This work provides an explanation for the high observed experimental efficiencies and bridges the knowledge gap between prior theoretical works and experiments by including the role of roughness in the electromagnetic near-field coupling in M-I-M TJs.