Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors

M. Shoaib, S. Bosch, O. Durmaz, Johan Scholten, Paul J.M. Havinga

    Research output: Contribution to journalArticleAcademicpeer-review

    108 Citations (Scopus)
    102 Downloads (Pure)

    Abstract

    The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.
    Original languageUndefined
    Pages (from-to)426:1-426:24
    Number of pages24
    JournalSensors (Switserland)
    Volume16
    Issue number4
    DOIs
    Publication statusPublished - 24 Mar 2016

    Keywords

    • smartwatch sensors
    • smoking recognition
    • EWI-26951
    • Sensor fusion
    • IR-100416
    • Body Worn Sensing
    • Gesture recognition
    • METIS-316896
    • Behavior analysis

    Cite this

    @article{cad32a409f54470290f093d3ce5de235,
    title = "Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors",
    abstract = "The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.",
    keywords = "smartwatch sensors, smoking recognition, EWI-26951, Sensor fusion, IR-100416, Body Worn Sensing, Gesture recognition, METIS-316896, Behavior analysis",
    author = "M. Shoaib and S. Bosch and O. Durmaz and Johan Scholten and Havinga, {Paul J.M.}",
    note = "eemcs-eprint-26951",
    year = "2016",
    month = "3",
    day = "24",
    doi = "10.3390/s16040426",
    language = "Undefined",
    volume = "16",
    pages = "426:1--426:24",
    journal = "Sensors (Switserland)",
    issn = "1424-8220",
    publisher = "Multidisciplinary Digital Publishing Institute",
    number = "4",

    }

    Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors. / Shoaib, M.; Bosch, S.; Durmaz, O.; Scholten, Johan; Havinga, Paul J.M.

    In: Sensors (Switserland), Vol. 16, No. 4, 24.03.2016, p. 426:1-426:24.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors

    AU - Shoaib, M.

    AU - Bosch, S.

    AU - Durmaz, O.

    AU - Scholten, Johan

    AU - Havinga, Paul J.M.

    N1 - eemcs-eprint-26951

    PY - 2016/3/24

    Y1 - 2016/3/24

    N2 - The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.

    AB - The position of on-body motion sensors plays an important role in human activity recognition. Most often, mobile phone sensors at the trouser pocket or an equivalent position are used for this purpose. However, this position is not suitable for recognizing activities that involve hand gestures, such as smoking, eating, drinking coffee and giving a talk. To recognize such activities, wrist-worn motion sensors are used. However, these two positions are mainly used in isolation. To use richer context information, we evaluate three motion sensors (accelerometer, gyroscope and linear acceleration sensor) at both wrist and pocket positions. Using three classifiers, we show that the combination of these two positions outperforms the wrist position alone, mainly at smaller segmentation windows. Another problem is that less-repetitive activities, such as smoking, eating, giving a talk and drinking coffee, cannot be recognized easily at smaller segmentation windows unlike repetitive activities, like walking, jogging and biking. For this purpose, we evaluate the effect of seven window sizes (2–30 s) on thirteen activities and show how increasing window size affects these various activities in different ways. We also propose various optimizations to further improve the recognition of these activities. For reproducibility, we make our dataset publicly available.

    KW - smartwatch sensors

    KW - smoking recognition

    KW - EWI-26951

    KW - Sensor fusion

    KW - IR-100416

    KW - Body Worn Sensing

    KW - Gesture recognition

    KW - METIS-316896

    KW - Behavior analysis

    U2 - 10.3390/s16040426

    DO - 10.3390/s16040426

    M3 - Article

    VL - 16

    SP - 426:1-426:24

    JO - Sensors (Switserland)

    JF - Sensors (Switserland)

    SN - 1424-8220

    IS - 4

    ER -