We present a custom-designed atomic force fluorescence microscope (AFFM), which can perform simultaneous optical and topographic measurements with single molecule sensitivity throughout the whole visible to near-infrared spectral region. Integration of atomic force microscopy (AFM) and confocal fluorescence microscopy combines the high-resolution topographical imaging of AFM with the reliable (bio)-chemical identification capability of optical methods. The AFFM is equipped with a spectrograph enabling combined topographic and fluorescence spectral imaging, which significantly enhances discrimination of spectroscopically distinct objects. The modular design allows easy switching between different modes of operation such as tip-scanning, sample-scanning or mechanical manipulation, all of which are combined with synchronous optical detection. We demonstrate that coupling the AFM with the fluorescence microscope does not compromise its ability to image with a high spatial resolution. Examples of several modes of operation of the AFFM are shown using two-dimensional crystals and membranes containing light-harvesting complexes from the photosynthetic bacterium Rhodobacter sphaeroides.