Can app changelogs improve requirements classification from app reviews? An exploratory study

Chong Wang, Fan Zhang, Peng Liang, Maya Daneva, Marten van Sinderen

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

2 Citations (Scopus)

Abstract

Background: Recent research on mining app reviews for software evolution indicated that the elicitation and analysis of user requirements can benefit from supplementing user reviews by data from other sources. However, only a few studies reported results of leveraging app changelogs together with app reviews.

Aims: Motivated by those findings, this exploratory experimental study looks into the role of app changelogs in the classification of requirements derived from app reviews. We aim at understanding if the use of app changelogs can lead to more accurate identification and classification of functional and non-functional requirements from app reviews. We also want to know which classification technique works better in this context.

Method: We did a case study on the effect of app changelogs on automatic classification of app reviews. Specifically, manual labeling, text preprocessing, and four supervised machine learning algorithms were applied to a series of experiments, varying in the number of app changelogs in the experimental data.

Results: We compared the accuracy of requirements classification from app reviews, by training the four classifiers with varying combinations of app reviews and changelogs. Among the four algorithms, Naïve Bayes was found to be more accurate for categorizing app reviews.

Conclusions: The results show that official app changelogs did not contribute to more accurate identification and classification of requirements from app reviews. In addition, Naïve Bayes seems to be more suitable for our further research on this topic.

Original languageEnglish
Title of host publicationProceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018
Place of PublicationPiscataway, NJ
PublisherIEEE Computer Society
Number of pages4
ISBN (Print)978-1-4503-5823-1
DOIs
Publication statusPublished - 11 Oct 2018
Event12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018 - Oulu, Finland
Duration: 11 Oct 201812 Oct 2018
Conference number: 12

Conference

Conference12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018
Abbreviated titleESEM
CountryFinland
CityOulu
Period11/10/1812/10/18

Fingerprint

Application programs
Labeling
Learning algorithms
Learning systems
Classifiers

Keywords

  • App changelogs
  • App reviews
  • Data-driven requirements engineering
  • Machine learning
  • Requirements analysis

Cite this

Wang, C., Zhang, F., Liang, P., Daneva, M., & van Sinderen, M. (2018). Can app changelogs improve requirements classification from app reviews? An exploratory study. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018 [a43] Piscataway, NJ: IEEE Computer Society. https://doi.org/10.1145/3239235.3267428
Wang, Chong ; Zhang, Fan ; Liang, Peng ; Daneva, Maya ; van Sinderen, Marten. / Can app changelogs improve requirements classification from app reviews? An exploratory study. Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018. Piscataway, NJ : IEEE Computer Society, 2018.
@inproceedings{16b20a2c5cd94a54b94bf178faad0972,
title = "Can app changelogs improve requirements classification from app reviews?: An exploratory study",
abstract = "Background: Recent research on mining app reviews for software evolution indicated that the elicitation and analysis of user requirements can benefit from supplementing user reviews by data from other sources. However, only a few studies reported results of leveraging app changelogs together with app reviews. Aims: Motivated by those findings, this exploratory experimental study looks into the role of app changelogs in the classification of requirements derived from app reviews. We aim at understanding if the use of app changelogs can lead to more accurate identification and classification of functional and non-functional requirements from app reviews. We also want to know which classification technique works better in this context.Method: We did a case study on the effect of app changelogs on automatic classification of app reviews. Specifically, manual labeling, text preprocessing, and four supervised machine learning algorithms were applied to a series of experiments, varying in the number of app changelogs in the experimental data. Results: We compared the accuracy of requirements classification from app reviews, by training the four classifiers with varying combinations of app reviews and changelogs. Among the four algorithms, Na{\"i}ve Bayes was found to be more accurate for categorizing app reviews.Conclusions: The results show that official app changelogs did not contribute to more accurate identification and classification of requirements from app reviews. In addition, Na{\"i}ve Bayes seems to be more suitable for our further research on this topic.",
keywords = "App changelogs, App reviews, Data-driven requirements engineering, Machine learning, Requirements analysis",
author = "Chong Wang and Fan Zhang and Peng Liang and Maya Daneva and {van Sinderen}, Marten",
year = "2018",
month = "10",
day = "11",
doi = "10.1145/3239235.3267428",
language = "English",
isbn = "978-1-4503-5823-1",
booktitle = "Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018",
publisher = "IEEE Computer Society",
address = "United States",

}

Wang, C, Zhang, F, Liang, P, Daneva, M & van Sinderen, M 2018, Can app changelogs improve requirements classification from app reviews? An exploratory study. in Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018., a43, IEEE Computer Society, Piscataway, NJ, 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018, Oulu, Finland, 11/10/18. https://doi.org/10.1145/3239235.3267428

Can app changelogs improve requirements classification from app reviews? An exploratory study. / Wang, Chong; Zhang, Fan; Liang, Peng; Daneva, Maya; van Sinderen, Marten.

Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018. Piscataway, NJ : IEEE Computer Society, 2018. a43.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - Can app changelogs improve requirements classification from app reviews?

T2 - An exploratory study

AU - Wang, Chong

AU - Zhang, Fan

AU - Liang, Peng

AU - Daneva, Maya

AU - van Sinderen, Marten

PY - 2018/10/11

Y1 - 2018/10/11

N2 - Background: Recent research on mining app reviews for software evolution indicated that the elicitation and analysis of user requirements can benefit from supplementing user reviews by data from other sources. However, only a few studies reported results of leveraging app changelogs together with app reviews. Aims: Motivated by those findings, this exploratory experimental study looks into the role of app changelogs in the classification of requirements derived from app reviews. We aim at understanding if the use of app changelogs can lead to more accurate identification and classification of functional and non-functional requirements from app reviews. We also want to know which classification technique works better in this context.Method: We did a case study on the effect of app changelogs on automatic classification of app reviews. Specifically, manual labeling, text preprocessing, and four supervised machine learning algorithms were applied to a series of experiments, varying in the number of app changelogs in the experimental data. Results: We compared the accuracy of requirements classification from app reviews, by training the four classifiers with varying combinations of app reviews and changelogs. Among the four algorithms, Naïve Bayes was found to be more accurate for categorizing app reviews.Conclusions: The results show that official app changelogs did not contribute to more accurate identification and classification of requirements from app reviews. In addition, Naïve Bayes seems to be more suitable for our further research on this topic.

AB - Background: Recent research on mining app reviews for software evolution indicated that the elicitation and analysis of user requirements can benefit from supplementing user reviews by data from other sources. However, only a few studies reported results of leveraging app changelogs together with app reviews. Aims: Motivated by those findings, this exploratory experimental study looks into the role of app changelogs in the classification of requirements derived from app reviews. We aim at understanding if the use of app changelogs can lead to more accurate identification and classification of functional and non-functional requirements from app reviews. We also want to know which classification technique works better in this context.Method: We did a case study on the effect of app changelogs on automatic classification of app reviews. Specifically, manual labeling, text preprocessing, and four supervised machine learning algorithms were applied to a series of experiments, varying in the number of app changelogs in the experimental data. Results: We compared the accuracy of requirements classification from app reviews, by training the four classifiers with varying combinations of app reviews and changelogs. Among the four algorithms, Naïve Bayes was found to be more accurate for categorizing app reviews.Conclusions: The results show that official app changelogs did not contribute to more accurate identification and classification of requirements from app reviews. In addition, Naïve Bayes seems to be more suitable for our further research on this topic.

KW - App changelogs

KW - App reviews

KW - Data-driven requirements engineering

KW - Machine learning

KW - Requirements analysis

UR - http://www.scopus.com/inward/record.url?scp=85061478238&partnerID=8YFLogxK

U2 - 10.1145/3239235.3267428

DO - 10.1145/3239235.3267428

M3 - Conference contribution

SN - 978-1-4503-5823-1

BT - Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018

PB - IEEE Computer Society

CY - Piscataway, NJ

ER -

Wang C, Zhang F, Liang P, Daneva M, van Sinderen M. Can app changelogs improve requirements classification from app reviews? An exploratory study. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2018. Piscataway, NJ: IEEE Computer Society. 2018. a43 https://doi.org/10.1145/3239235.3267428