MultEYE: Monitoring system for real-time vehicle detection, tracking and speed estimation from UAV imagery on edge-computing platforms

N. Balamuralidhar, S. Tilon, F. Nex

Research output: Contribution to journalArticleAcademicpeer-review

58 Citations (Scopus)
161 Downloads (Pure)


We present MultEYE, a traffic monitoring system that can detect, track, and estimate the velocity of vehicles in a sequence of aerial images. The presented solution has been optimized to execute these tasks in real-time on an embedded computer installed on an Unmanned Aerial Vehicle (UAV). In order to overcome the limitation of existing object detection architectures related to accuracy and computational overhead, a multi-task learning methodology was employed by adding a segmentation head to an object detector backbone resulting in the MultEYE object detection architecture. On a custom dataset, it achieved 4.8% higher mean Average Precision (mAP) score, while being 91.4% faster than the state-of-the-art model and while being able to generalize to different real-world traffic scenes. Dedicated object tracking and speed estimation algorithms have been then optimized to track reliably objects from an UAV with limited computational effort. Different strategies to combine object detection, tracking, and speed estimation are discussed, too. From our experiments, the optimized detector runs at an average frame-rate of up to 29 frames per second (FPS) on frame resolution 512 × 320 on a Nvidia Xavier NX board, while the optimally combined detector, tracker and speed estimator pipeline achieves speeds of up to 33 FPS on an image of resolution 3072 × 1728. To our knowledge, the MultEYE system is one of the first traffic monitoring systems that was specifically designed and optimized for an UAV platform under real-world constraints.

Original languageEnglish
Article number573
Pages (from-to)1-24
Number of pages24
JournalRemote sensing
Issue number4
Publication statusPublished - 5 Feb 2021


  • Edge computing
  • Multi-task learning
  • Object detection
  • Segmentation
  • Traffic monitoring
  • Unmanned Aerial Vehicles
  • Vehicle detection
  • Vehicle tracking


Dive into the research topics of 'MultEYE: Monitoring system for real-time vehicle detection, tracking and speed estimation from UAV imagery on edge-computing platforms'. Together they form a unique fingerprint.

Cite this