Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors

M. Shoaib, S. Bosch, O. Durmaz, Johan Scholten, Paul J.M. Havinga

Research output: Contribution to journalArticle

  • 41 Citations

Abstract

The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.
LanguageUndefined
Pages426:1-426:24
Number of pages24
JournalSensors (Switserland)
Volume16
Issue number4
DOIs
StatePublished - 24 Mar 2016

Keywords

  • smartwatch sensors
  • smoking recognition
  • EWI-26951
  • Sensor fusion
  • IR-100416
  • Body Worn Sensing
  • Gesture recognition
  • METIS-316896
  • Behavior analysis

Cite this

@article{cad32a409f54470290f093d3ce5de235,
title = "Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors",
abstract = "The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.",
keywords = "smartwatch sensors, smoking recognition, EWI-26951, Sensor fusion, IR-100416, Body Worn Sensing, Gesture recognition, METIS-316896, Behavior analysis",
author = "M. Shoaib and S. Bosch and O. Durmaz and Johan Scholten and Havinga, {Paul J.M.}",
note = "eemcs-eprint-26951",
year = "2016",
month = "3",
day = "24",
doi = "10.3390/s16040426",
language = "Undefined",
volume = "16",
pages = "426:1--426:24",
journal = "Sensors (Switserland)",
issn = "1424-8220",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "4",

}

Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors. / Shoaib, M.; Bosch, S.; Durmaz, O.; Scholten, Johan; Havinga, Paul J.M.

In: Sensors (Switserland), Vol. 16, No. 4, 24.03.2016, p. 426:1-426:24.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors

AU - Shoaib,M.

AU - Bosch,S.

AU - Durmaz,O.

AU - Scholten,Johan

AU - Havinga,Paul J.M.

N1 - eemcs-eprint-26951

PY - 2016/3/24

Y1 - 2016/3/24

N2 - The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.

AB - The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.

KW - smartwatch sensors

KW - smoking recognition

KW - EWI-26951

KW - Sensor fusion

KW - IR-100416

KW - Body Worn Sensing

KW - Gesture recognition

KW - METIS-316896

KW - Behavior analysis

U2 - 10.3390/s16040426

DO - 10.3390/s16040426

M3 - Article

VL - 16

SP - 426:1-426:24

JO - Sensors (Switserland)

T2 - Sensors (Switserland)

JF - Sensors (Switserland)

SN - 1424-8220

IS - 4

ER -