Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks

Nils Witt; Christin Seifert

doi:10.1007/978-3-319-67008-9_16

Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks

Nils Witt, Christin Seifert

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

5 Citations (Scopus)

Abstract

Many applications in the natural language processing domain require the tuning of machine learning algorithms, which involves adaptation of hyperparameters. We perform experiments by systematically varying hyperparameter settings of text embedding algorithms to obtain insights about the influence and interrelation of hyperparameters on the model performance on a text classification task using text embedding features. For some parameters (e.g., size of the context window) we could not find an influence on the accuracy while others (e.g., dimensionality of the embeddings) strongly influence the results, but have a range where the results are nearly optimal. These insights are beneficial to researchers and practitioners in order to find sensible hyperparameter configurations for research projects based on text embeddings. This reduces the parameter search space and the amount of (manual and automatic) optimization time.

Original language	English
Title of host publication	Research and Advanced Technology for Digital Libraries
Subtitle of host publication	21st International Conference on Theory and Practice of Digital Libraries, TPDL 2017, Thessaloniki, Greece, September 18-21, 2017, Proceedings
Editors	Jaap Kamps, Giannis Tsakonas, Yannis Manolopoulos, Lazaros Iliadis, Ioannis Karydis
Publisher	Springer
Pages	193-204
ISBN (Electronic)	978-3-319-67008-9
ISBN (Print)	978-3-319-67007-2
DOIs	https://doi.org/10.1007/978-3-319-67008-9_16
Publication status	Published - 2017
Event	21st International Conference on Theory and Practice of Digital Libraries 2017 - Grand Hotel Palace, Thessaloniki, Thessaloniki, Greece Duration: 18 Sep 2017 → 21 Sep 2017 Conference number: 21 http://www.tpdl.eu/tpdl2017/

Publication series

Name	Lecture Notes in Computer Science
Volume	10450

Conference

Conference	21st International Conference on Theory and Practice of Digital Libraries 2017
Abbreviated title	TPDL 2017
Country	Greece
City	Thessaloniki
Period	18/09/17 → 21/09/17
Internet address	http://www.tpdl.eu/tpdl2017/

Access to Document

10.1007/978-3-319-67008-9_16

Fingerprint Dive into the research topics of 'Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks'. Together they form a unique fingerprint.

Cite this

Witt, N., & Seifert, C. (2017). Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks. In J. Kamps, G. Tsakonas, Y. Manolopoulos, L. Iliadis, & I. Karydis (Eds.), Research and Advanced Technology for Digital Libraries: 21st International Conference on Theory and Practice of Digital Libraries, TPDL 2017, Thessaloniki, Greece, September 18-21, 2017, Proceedings (pp. 193-204). (Lecture Notes in Computer Science; Vol. 10450). Springer. https://doi.org/10.1007/978-3-319-67008-9_16

Witt, Nils ; Seifert, Christin. / Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks. Research and Advanced Technology for Digital Libraries: 21st International Conference on Theory and Practice of Digital Libraries, TPDL 2017, Thessaloniki, Greece, September 18-21, 2017, Proceedings. editor / Jaap Kamps ; Giannis Tsakonas ; Yannis Manolopoulos ; Lazaros Iliadis ; Ioannis Karydis. Springer, 2017. pp. 193-204 (Lecture Notes in Computer Science).

@inproceedings{a1b29ea9a3824c328c79c697d4a14a38,

title = "Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks",

abstract = "Many applications in the natural language processing domain require the tuning of machine learning algorithms, which involves adaptation of hyperparameters. We perform experiments by systematically varying hyperparameter settings of text embedding algorithms to obtain insights about the influence and interrelation of hyperparameters on the model performance on a text classification task using text embedding features. For some parameters (e.g., size of the context window) we could not find an influence on the accuracy while others (e.g., dimensionality of the embeddings) strongly influence the results, but have a range where the results are nearly optimal. These insights are beneficial to researchers and practitioners in order to find sensible hyperparameter configurations for research projects based on text embeddings. This reduces the parameter search space and the amount of (manual and automatic) optimization time.",

author = "Nils Witt and Christin Seifert",

year = "2017",

doi = "10.1007/978-3-319-67008-9_16",

language = "English",

isbn = "978-3-319-67007-2",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "193--204",

editor = "Jaap Kamps and Giannis Tsakonas and Yannis Manolopoulos and Lazaros Iliadis and Ioannis Karydis",

booktitle = "Research and Advanced Technology for Digital Libraries",

address = "Netherlands",

note = "21st International Conference on Theory and Practice of Digital Libraries 2017, TPDL 2017 ; Conference date: 18-09-2017 Through 21-09-2017",

url = "http://www.tpdl.eu/tpdl2017/",

}

Witt, N & Seifert, C 2017, Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks. in J Kamps, G Tsakonas, Y Manolopoulos, L Iliadis & I Karydis (eds), Research and Advanced Technology for Digital Libraries: 21st International Conference on Theory and Practice of Digital Libraries, TPDL 2017, Thessaloniki, Greece, September 18-21, 2017, Proceedings. Lecture Notes in Computer Science, vol. 10450, Springer, pp. 193-204, 21st International Conference on Theory and Practice of Digital Libraries 2017, Thessaloniki, Greece, 18/09/17. https://doi.org/10.1007/978-3-319-67008-9_16

Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks. / Witt, Nils; Seifert, Christin.

Research and Advanced Technology for Digital Libraries: 21st International Conference on Theory and Practice of Digital Libraries, TPDL 2017, Thessaloniki, Greece, September 18-21, 2017, Proceedings. ed. / Jaap Kamps; Giannis Tsakonas; Yannis Manolopoulos; Lazaros Iliadis; Ioannis Karydis. Springer, 2017. p. 193-204 (Lecture Notes in Computer Science; Vol. 10450).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks

AU - Witt, Nils

AU - Seifert, Christin

N1 - Conference code: 21

PY - 2017

Y1 - 2017

N2 - Many applications in the natural language processing domain require the tuning of machine learning algorithms, which involves adaptation of hyperparameters. We perform experiments by systematically varying hyperparameter settings of text embedding algorithms to obtain insights about the influence and interrelation of hyperparameters on the model performance on a text classification task using text embedding features. For some parameters (e.g., size of the context window) we could not find an influence on the accuracy while others (e.g., dimensionality of the embeddings) strongly influence the results, but have a range where the results are nearly optimal. These insights are beneficial to researchers and practitioners in order to find sensible hyperparameter configurations for research projects based on text embeddings. This reduces the parameter search space and the amount of (manual and automatic) optimization time.

AB - Many applications in the natural language processing domain require the tuning of machine learning algorithms, which involves adaptation of hyperparameters. We perform experiments by systematically varying hyperparameter settings of text embedding algorithms to obtain insights about the influence and interrelation of hyperparameters on the model performance on a text classification task using text embedding features. For some parameters (e.g., size of the context window) we could not find an influence on the accuracy while others (e.g., dimensionality of the embeddings) strongly influence the results, but have a range where the results are nearly optimal. These insights are beneficial to researchers and practitioners in order to find sensible hyperparameter configurations for research projects based on text embeddings. This reduces the parameter search space and the amount of (manual and automatic) optimization time.

U2 - 10.1007/978-3-319-67008-9_16

DO - 10.1007/978-3-319-67008-9_16

M3 - Conference contribution

SN - 978-3-319-67007-2

T3 - Lecture Notes in Computer Science

SP - 193

EP - 204

BT - Research and Advanced Technology for Digital Libraries

A2 - Kamps, Jaap

A2 - Tsakonas, Giannis

A2 - Manolopoulos, Yannis

A2 - Iliadis, Lazaros

A2 - Karydis, Ioannis

PB - Springer

T2 - 21st International Conference on Theory and Practice of Digital Libraries 2017

Y2 - 18 September 2017 through 21 September 2017

ER -

Witt N, Seifert C. Understanding the Influence of Hyperparameters on Text Embeddings for Text Classification Tasks. In Kamps J, Tsakonas G, Manolopoulos Y, Iliadis L, Karydis I, editors, Research and Advanced Technology for Digital Libraries: 21st International Conference on Theory and Practice of Digital Libraries, TPDL 2017, Thessaloniki, Greece, September 18-21, 2017, Proceedings. Springer. 2017. p. 193-204. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-319-67008-9_16