Surfaces covered with end-anchored polymers under good solvent conditions have excellent tribological properties. The friction between such surfaces is commonly attributed to steady-state interdigitation of the opposing polymer brushes. However, this conclusion tends to be based on idealized geometries neglecting surface roughness. Using molecular dynamics simulations, we find that there are qualitative differences between the friction of rough and flat polymer-brush surfaces. For rough surfaces the dissipation due to transient interdigitation and capillary- and shape-hysteresis is just as important or can even dominate over steady-state interdigitation. Having a mix of dissipation mechanisms that are all intertwined affects the observed friction force in linear-response as well as in the shear-thinning exponents and effective viscosity. Moreover, we find that the effect of the solvent viscosity is sublinear.