In most of the tribological contacts, the composition and tribological properties of the original interface will change during use. The tribo-films, with modified properties compared to the bulk, are dynamic structures that play a significant role in friction. The existence of a tribo-modified surface layer and its importance on the overall friction of elastomers has been shown both theoretically and experimentally before. The characteristics of the modified surface layer deserve specific attention since the tribological properties of elastomers in contact with a rough counter-surface are determined by these modified surfaces together with the properties of bulk of the material. Both the formation of the modified layer and the break down (wear) of it are of importance in determining the existence and thickness of the tribo-modified layer. In this study, the importance of the wear is emphasized by comparing two styrene butadiene rubber-based elastomers in contact with a granite sphere. A current status of perception of the removal and the stability of the modified surface layers on rubbers is introduced as well as experimental work related to this matter and discussion within literature. Pin-on-disk friction tests are performed on two SBR-based samples in contact with a granite sphere under controlled environmental conditions to form the modified surface layer. Although the hysteresis part of the friction force which has a minor contribution in the overall friction is not markedly different, the total measured friction coefficient differs significantly. Mechanical changes both inside and outside the wear track are determined by atomic force microscope nano-indentations at different timescales to examine the modified surface layer on the test samples. The specific wear rates of the two tribo-systems are compared, and the existence of the modified surface layer, the different measured friction coefficient and the running-in distances toward steady-state friction are explained considering different wear rates. A conceptual model is presented, correlating the energy input into the tribo-system and the existence of a modified surface layer.