Free-electron lasers (FELs) have been built ranging in wavelength from long-wavelength oscillators using partial wave guiding through ultraviolet through hard x-ray that are either seeded or start from noise. In addition, FELs that produce different polarizations of the output radiation ranging from linear through elliptic to circular polarization are currently under study. In this paper, we develop a three-dimensional, time-dependent formulation that is capable of modeling this large variety of FEL configurations including different polarizations. We employ a modal expansion for the optical field, i.e., a Gaussian expansion with variable polarization for free-space propagation. This formulation uses the full Newton–Lorentz force equations to track the particles through the optical and magnetostatic fields. As a result, arbitrary three-dimensional representations for different undulator configurations are implemented, including planar, helical, and elliptical undulators. In particular, we present an analytic model of an APPLE-II undulator to treat arbitrary elliptical polarizations, which is used to treat general elliptical polarizations. To model oscillator configurations, and allow propagation of the optical field outside the undulator and interact with optical elements, we link the FEL simulation with the optical propagation code OPC. We present simulations using the APPLE-II undulator model to produce elliptically polarized output radiation, and present a detailed comparison with recent experiments using a tapered undulator configuration at the Linac Coherent Light Source. Validation of the nonlinear formation is also shown by comparison with experimental results obtained in the Sorgente Pulsata Auto-amplificata di Radiazione Coerente SASE FEL experiment at ENEA Frascati, a seeded tapered amplifier experiment at Brookhaven National Laboratory, and the 10 kW upgrade oscillator experiment at the Thomas Jefferson National Accelerator Facility.