Vibro-acoustic modulation–based damage identification relies on the modulation of a high-frequency carrier signal by an intenser low-frequency vibration signal due to damage-induced structural nonlinearities. A time domain analysis of the vibro-acoustic modulation phenomena was presented at multiple spatial locations in an impact damaged composite skin–stiffener structure. The instantaneous amplitude and frequency of the carrier velocity response were extracted to analyze the intermodulation effects between the two excitation signals. Increased amplitude modulations at the damaged region revealed the presence, location, and length of the skin–stiffener damage. The damage hardly modulated the frequency of the carrier response. This difference in behavior was attributed to the nonlinear skin–stiffener interaction introduced by the periodic opening and closing of the damage, according to earlier research by authors on the same structure. A parametric study showed that the amplitude and phase of the amplitude modulation are dependent on the selected carrier excitation frequency, and hence the high-frequency wave field that is introduced. This work demonstrates not only the potential but also the complexity of the vibro-acoustic modulation based damage identification approach.