Interactive multi-object tracking for virtual object manipulation

Yibo Guo, Michael Ying Yang*, Bodo Rosenhahn

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review


We present an interactive system to manipulate a virtual object by tracking multiple hands in 3D space using a Kinect device. The system segments hand shapes from a captured 3D scene by using depth information and active contours. A hand shape is recognized by a trained naive Bayes classifier, whether it belongs to a palm, a pointing hand form or both hands with simple occlusion. A plane is approximated by using RANSAC for a palm hand form, while a vector from the hand centroid to the fingertip is obtained for a pointing hand form by using ICP as tracking method. A shape of simple occluded two hands is split into a palm and an incomplete pointing hand form, whose missing data is estimated by using PCA. The system works in semi-real time.

Original languageEnglish
Title of host publicationISA13 – The ISPRS Workshop on Image Sequence Analysis 2013 (Volume II-3/W2)
Subtitle of host publication11 November 2013, Antalya, Turkey
EditorsC. Mallet, A. Yilmaz, Y. Vizilter, M.Y. Yang
PublisherInternational Society for Photogrammetry and Remote Sensing (ISPRS)
Number of pages6
Publication statusPublished - 2 Oct 2013
Externally publishedYes
Event2013 ISPRS Workshop on Image Sequence Analysis, ISA 2013 - Antalya, Turkey
Duration: 11 Nov 201311 Nov 2013

Publication series

NameISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
ISSN (Print)2194-9042


Conference2013 ISPRS Workshop on Image Sequence Analysis, ISA 2013
Abbreviated titleISA 2013


  • 1D signature
  • Gesture recognition
  • ICP
  • Kinect
  • Multiple object tracking


Dive into the research topics of 'Interactive multi-object tracking for virtual object manipulation'. Together they form a unique fingerprint.

Cite this