The creation of a photon-atom bound state was first envisaged for the case of an atom in a long-lived excited state inside a high-quality microwave cavity. In practice, however, light forces in the microwave domain are insufficient to support an atom against gravity. Although optical photons can provide forces of the required magnitude, atomic decay rates and cavity losses are larger too, and so the atom-cavity system must be continually excited by an external laser. Such an approach also permits continuous observation of the atom's position, by monitoring the light transmitted through the cavity. The dual role of photons in this system distinguishes it from other single-atom experiments such as those using magneto-optical traps, ion traps or a far-off-resonance optical trap. Here we report high-finesse optical cavity experiments in which the change in transmission induced by a single slow atom approaching the cavity triggers an external feedback switch which traps the atom in a light field containing about one photon on average. The oscillatory motion of the trapped atom induces oscillations in the transmitted light intensity; we attribute periodic structure in intensity- correlation-function data to 'long-distance' flights of the atom between different anti-nodes of the standing-wave in the cavity. The system should facilitate investigations of the dynamics of single quantum objects and may find future applications in quantum information processing.