Opening the black box of eHealth: Collecting, analyzing, and interpreting log data

Research output: Contribution to journalArticleAcademicpeer-review

56 Downloads (Pure)

Abstract

Background: In eHealth research, limited insights have been obtained on process outcomes or how the use of technology has contributed to the users’ ability to have a healthier life, improved wellbeing, or activate new attitudes in their daily tasks. As a result, eHealth is often perceived as a black box. To open this Black Box of eHealth, methodologies must extend beyond the classic effect evaluations. The analyses of log data (anonymous records of real-time actions performed by each user) can provide continuous and objective insights into the actual usage of the technology. However, until now the possibilities of log data in eHealth research has not been exploited to its fullest extent.
Objectives: The aim of this paper is to describe how log data can be used to improve the evaluation and understand the use of an eHealth technology with a broader approach than only descriptive statistics. This paper serves as a starting point for using log data analysis in eHealth research.
Methods: First, we describe what log data is and an overview is given of research questions to evaluate the system, the context, the users of a technology as well as the underpinning theoretical constructs. Secondly, requirements for log data, the starting points for the data preparation and methods for data collection are explained.
Results: In the third part, some methods for data analysis are described. Finally, a conclusion is drawn regarding the importance of the results for both scientific and practical applications.
Conclusion: The analysis of log data can be of great value for opening the black box of eHealth. A deliberate log data analysis can give new insights into how the usage of the technology contributed to the found effects and can thereby help to improve the persuasiveness and effectiveness of the eHealth technology and the underpinning behavioral models.
Keywords: eHealth, black box, evaluation, log data analysis
Original languageEnglish
Article numbere156
JournalJMIR research protocols
Volume6
Issue number8
DOIs
Publication statusPublished - Aug 2017

Fingerprint

Statistics

Cite this

@article{2e842e9ada54452cbfee87d64a17ac6a,
title = "Opening the black box of eHealth: Collecting, analyzing, and interpreting log data",
abstract = "Background: In eHealth research, limited insights have been obtained on process outcomes or how the use of technology has contributed to the users’ ability to have a healthier life, improved wellbeing, or activate new attitudes in their daily tasks. As a result, eHealth is often perceived as a black box. To open this Black Box of eHealth, methodologies must extend beyond the classic effect evaluations. The analyses of log data (anonymous records of real-time actions performed by each user) can provide continuous and objective insights into the actual usage of the technology. However, until now the possibilities of log data in eHealth research has not been exploited to its fullest extent. Objectives: The aim of this paper is to describe how log data can be used to improve the evaluation and understand the use of an eHealth technology with a broader approach than only descriptive statistics. This paper serves as a starting point for using log data analysis in eHealth research. Methods: First, we describe what log data is and an overview is given of research questions to evaluate the system, the context, the users of a technology as well as the underpinning theoretical constructs. Secondly, requirements for log data, the starting points for the data preparation and methods for data collection are explained. Results: In the third part, some methods for data analysis are described. Finally, a conclusion is drawn regarding the importance of the results for both scientific and practical applications.Conclusion: The analysis of log data can be of great value for opening the black box of eHealth. A deliberate log data analysis can give new insights into how the usage of the technology contributed to the found effects and can thereby help to improve the persuasiveness and effectiveness of the eHealth technology and the underpinning behavioral models.Keywords: eHealth, black box, evaluation, log data analysis",
author = "Floor Sieverink and Kelders, {Saskia Marion} and Mannes Poel and {van Gemert-Pijnen}, {Julia E.W.C.}",
year = "2017",
month = "8",
doi = "10.2196/resprot.6452",
language = "English",
volume = "6",
journal = "JMIR research protocols",
issn = "1929-0748",
publisher = "JMIR Publications",
number = "8",

}

Opening the black box of eHealth: Collecting, analyzing, and interpreting log data. / Sieverink, Floor ; Kelders, Saskia Marion; Poel, Mannes ; van Gemert-Pijnen, Julia E.W.C.

In: JMIR research protocols, Vol. 6, No. 8, e156, 08.2017.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Opening the black box of eHealth: Collecting, analyzing, and interpreting log data

AU - Sieverink, Floor

AU - Kelders, Saskia Marion

AU - Poel, Mannes

AU - van Gemert-Pijnen, Julia E.W.C.

PY - 2017/8

Y1 - 2017/8

N2 - Background: In eHealth research, limited insights have been obtained on process outcomes or how the use of technology has contributed to the users’ ability to have a healthier life, improved wellbeing, or activate new attitudes in their daily tasks. As a result, eHealth is often perceived as a black box. To open this Black Box of eHealth, methodologies must extend beyond the classic effect evaluations. The analyses of log data (anonymous records of real-time actions performed by each user) can provide continuous and objective insights into the actual usage of the technology. However, until now the possibilities of log data in eHealth research has not been exploited to its fullest extent. Objectives: The aim of this paper is to describe how log data can be used to improve the evaluation and understand the use of an eHealth technology with a broader approach than only descriptive statistics. This paper serves as a starting point for using log data analysis in eHealth research. Methods: First, we describe what log data is and an overview is given of research questions to evaluate the system, the context, the users of a technology as well as the underpinning theoretical constructs. Secondly, requirements for log data, the starting points for the data preparation and methods for data collection are explained. Results: In the third part, some methods for data analysis are described. Finally, a conclusion is drawn regarding the importance of the results for both scientific and practical applications.Conclusion: The analysis of log data can be of great value for opening the black box of eHealth. A deliberate log data analysis can give new insights into how the usage of the technology contributed to the found effects and can thereby help to improve the persuasiveness and effectiveness of the eHealth technology and the underpinning behavioral models.Keywords: eHealth, black box, evaluation, log data analysis

AB - Background: In eHealth research, limited insights have been obtained on process outcomes or how the use of technology has contributed to the users’ ability to have a healthier life, improved wellbeing, or activate new attitudes in their daily tasks. As a result, eHealth is often perceived as a black box. To open this Black Box of eHealth, methodologies must extend beyond the classic effect evaluations. The analyses of log data (anonymous records of real-time actions performed by each user) can provide continuous and objective insights into the actual usage of the technology. However, until now the possibilities of log data in eHealth research has not been exploited to its fullest extent. Objectives: The aim of this paper is to describe how log data can be used to improve the evaluation and understand the use of an eHealth technology with a broader approach than only descriptive statistics. This paper serves as a starting point for using log data analysis in eHealth research. Methods: First, we describe what log data is and an overview is given of research questions to evaluate the system, the context, the users of a technology as well as the underpinning theoretical constructs. Secondly, requirements for log data, the starting points for the data preparation and methods for data collection are explained. Results: In the third part, some methods for data analysis are described. Finally, a conclusion is drawn regarding the importance of the results for both scientific and practical applications.Conclusion: The analysis of log data can be of great value for opening the black box of eHealth. A deliberate log data analysis can give new insights into how the usage of the technology contributed to the found effects and can thereby help to improve the persuasiveness and effectiveness of the eHealth technology and the underpinning behavioral models.Keywords: eHealth, black box, evaluation, log data analysis

U2 - 10.2196/resprot.6452

DO - 10.2196/resprot.6452

M3 - Article

VL - 6

JO - JMIR research protocols

JF - JMIR research protocols

SN - 1929-0748

IS - 8

M1 - e156

ER -