Traffic control recognition with an attention mechanism using speed-profile and satellite imagery data

raffic regulators at intersections act as an essential factor that influences traffic flow and, subsequently, the route choices of commuters. A digital map that provides up-to-date traffic control information is beneficial not only for facilitating the commuters’ trips, but also for energy-saving and environmental protection. In this paper, instead of using expensive surveying methods, we propose an automatic way based on a Conditional Variational Autoencoder (CVAE) to recognize traffic regulators, i. e., arm rules at intersections, by leveraging the GPS data collected from vehicles and the satellite imagery retrieved from digital maps, i. e., Google Maps. We apply a Long Short-Term Memory to extract the motion dynamics over a GPS sequence traversed through the intersection. Simultaneously, we build a Convolutional Neural Network (CNN) to extract the grid-based local imagery information associated with each step of the GPS positions. Moreover, a self-attention mechanism is adopted to extract the spatial and temporal features over both the GPS and grid sequences. The extracted temporal and spatial features are then combined for detecting the traffic arm rules. To analyze the performance of our method, we tested it on a GPS dataset collected by driving vehicles in Hannover, a medium-sized German city. Compared to a Random Forest model and an Encoder-Decoder model, our proposed model achieved better results with both accuracy and F1-score of 0.90 for the three-class (arm rules of uncontrolled, traffic light, and priority sign) task. We also carried out ablation studies to further investigate the effectiveness of the GPS input branch, the image input branch, and the self-attention mechanism in our model.