Face super resolution with a high frequency highway

Face shape priors such as landmarks, heatmaps, and parsing maps are widely used to improve face super resolution (SR). It is observed that face priors provide locations of high-frequency details in key facial areas such as the eyes and mouth. However, existing methods fail to effectively exploit the high-frequency information by using the priors as either constraints or inputs. This paper proposes a novel high frequency highway ((Formula presented.)) framework to better utilize prior information for face SR, which dynamically decomposes the final SR face into a coarse SR face and a high frequency (HF) face. The coarse SR face is reconstructed from a low-resolution face via a texture branch, using only pixel-wise reconstruction loss. Meanwhile, the HF face is directly generated from face priors via an HF branch that employs the proposed inception–hourglass model. As a result, (Formula presented.) allows the face priors to have a direct impact on the SR face by adding the outputs of both branches as the final result and provides an extra face editing function. Extensive experiments show that (Formula presented.) significantly outperforms state-of-the-art face SR methods, is general for different texture branch models and face priors, and is robust to dataset mismatch and pose variations.