Transparent conducting oxides (TCOs) have recently attracted much attention in the fields of optics and photonics because of their outstanding linear and nonlinear optical response in the near-zero permittivity spectral region. The optical response of these materials can be further enhanced by optimizing the material properties through fabrication. In particular, two important TCO parameters affecting the strength of the optical interactions are the optical mobility and effective mass of free electrons. In this work, we fabricate epitaxial, highly-textured, and polycrystalline tin-doped indium oxide (ITO) films to experimentally study the effect of the crystal quality on the optical mobility and effective electron mass, and on the optical properties of the material in the near-zero permittivity spectral region. Compared to polycrystalline ITO, we report an increase in the optical mobility from 38 to 67 cm2/Vs and a reduction in the effective mass from 0.3 m0 to 0.24 m0 in oxygen-deficient epitaxially grown ITO films. The improved material parameters reduces the imaginary part of the permittivity (from 0.56 to 0.42) and results in a steeper material dispersion for the high-crystal-quality ITO films. Based on these results, an analysis of the figure of merit for nonlinear refraction reveals that epi- and tex-ITO films can achieve a stronger nonlinear response than poly-ITO samples. Our results show that controlling the free-electron optical mobility and effective mass through crystal quality along with tuning the free-electron density allows for tailoring simultaneously the near-zero-permittivity wavelength and the optical losses at that wavelength, which is of utmost importance for the ENZ photonics applications.