Traffic control recognition with an attention mechanism using speed-profile and satellite imagery data

Hao Cheng, Haoran Lei, Stefania Zourlidou, Monika Sester

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

19 Downloads (Pure)


raffic regulators at intersections act as an essential factor that influences traffic flow and, subsequently, the route choices of commuters. A digital map that provides up-to-date traffic control information is beneficial not only for facilitating the commuters’ trips, but also for energy-saving and environmental protection. In this paper, instead of using expensive surveying methods, we propose an automatic way based on a Conditional Variational Autoencoder (CVAE) to recognize traffic regulators, i. e., arm rules at intersections, by leveraging the GPS data collected from vehicles and the satellite imagery retrieved from digital maps, i. e., Google Maps. We apply a Long Short-Term Memory to extract the motion dynamics over a GPS sequence traversed through the intersection. Simultaneously, we build a Convolutional Neural Network (CNN) to extract the grid-based local imagery information associated with each step of the GPS positions. Moreover, a self-attention mechanism is adopted to extract the spatial and temporal features over both the GPS and grid sequences. The extracted temporal and spatial features are then combined for detecting the traffic arm rules. To analyze the performance of our method, we tested it on a GPS dataset collected by driving vehicles in Hannover, a medium-sized German city. Compared to a Random Forest model and an Encoder-Decoder model, our proposed model achieved better results with both accuracy and F1-score of 0.90 for the three-class (arm rules of uncontrolled, traffic light, and priority sign) task. We also carried out ablation studies to further investigate the effectiveness of the GPS input branch, the image input branch, and the self-attention mechanism in our model.
Original languageEnglish
Title of host publicationXXIV ISPRS Congress “Imaging today, foreseeing tomorrow”
Subtitle of host publicationCommission IV
EditorsS. Zlatanova, G. Sithole, J. Barton
Number of pages7
Publication statusPublished - 1 Jun 2022
Externally publishedYes
EventXXIV ISPRS Congress 2022: Congress “Imaging today, foreseeing tomorrow” - Nice, France
Duration: 6 Jun 202211 Jun 2022

Publication series

NameInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
ISSN (Print)1682-1750


ConferenceXXIV ISPRS Congress 2022
Abbreviated titleISPRS 2022
Internet address


  • ITC-CV


Dive into the research topics of 'Traffic control recognition with an attention mechanism using speed-profile and satellite imagery data'. Together they form a unique fingerprint.

Cite this