Localization and classification of overlapping sound events based on spectrogram-keypoint using acoustic-sensor-network data

Automatic localization and classification of environmental sound events can provide great aid to many human-centric IoT applications. However as many papers have mentioned, environmental sound events in daily life are complicated and hard to classify especially when multiple sounds happen simultaneously. Unlike most works, which decompose and classify overlapping signals using a unified model, we first decompose overlapping sound signals with a spectrogram-keypoint based localization algorithm. These located and clustered spectrogram-keypoints are subsequently reused for sound source classification. Our major contribution is the modeling of a global cost function to synchronize the time-difference-of-arrivals (TDOA) of each small spectrogram-keypoint and further locating the sound sources by these clustered keypoints. With these clustered keypoints, 2 different classification models are used to classify the sound sources. Our experiments show that our solution is both accurate and low-cost in terms of calculation effort.