Word level language identification in online multilingual communication

Dong-Phuong Nguyen; A. Seza Dogruoz

Word level language identification in online multilingual communication

Dong-Phuong Nguyen, A. Seza Dogruoz

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

86 Citations (Scopus)

50 Downloads (Pure)

Abstract

Multilingual speakers switch between languages in online and spoken communication. Analyses of large scale multilingual data require automatic language identification at the word level. For our experiments with multilingual online discussions, we first tag the language of individual words using language models and dictionaries. Secondly, we incorporate context to improve the performance. We achieve an accuracy of 98%. Besides word level accuracy, we use two new metrics to evaluate this task.

Original language	Undefined
Title of host publication	Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
Place of Publication	Michigan, USA
Publisher	Association for Computational Linguistics (ACL)
Pages	857-862
Number of pages	6
ISBN (Print)	978-1-937284-97-8
Publication status	Published - 18 Oct 2013
Event	Conference on Empirical Methods in Natural Language Processing 2013 - Grand Hyatt Seattle, Seattle, United States Duration: 18 Oct 2013 → 21 Oct 2013 http://mirror.aclweb.org/emnlp2013/

Publication series

Name
Publisher	Association for Computational Linguistics

Conference

Conference	Conference on Empirical Methods in Natural Language Processing 2013
Abbreviated title	EMNLP 2013
Country/Territory	United States
City	Seattle
Period	18/10/13 → 21/10/13
Internet address	http://mirror.aclweb.org/emnlp2013/

Keywords

Language identification
multilingual
METIS-302564
EWI-24092
IR-88555
Social Media

Access to Document

D13-1084

http://aclweb.org/anthology//D/D13/D13-1084.pdf

Cite this

@inproceedings{23fcc7f84fd041ed827f51d17f745e4c,

title = "Word level language identification in online multilingual communication",

abstract = "Multilingual speakers switch between languages in online and spoken communication. Analyses of large scale multilingual data require automatic language identification at the word level. For our experiments with multilingual online discussions, we first tag the language of individual words using language models and dictionaries. Secondly, we incorporate context to improve the performance. We achieve an accuracy of 98%. Besides word level accuracy, we use two new metrics to evaluate this task.",

keywords = "Language identification, multilingual, METIS-302564, EWI-24092, IR-88555, Social Media",

author = "Dong-Phuong Nguyen and Dogruoz, {A. Seza}",

year = "2013",

month = oct,

day = "18",

language = "Undefined",

isbn = "978-1-937284-97-8",

publisher = "Association for Computational Linguistics (ACL)",

pages = "857--862",

booktitle = "Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing",

address = "United States",

note = "Conference on Empirical Methods in Natural Language Processing 2013 ; Conference date: 18-10-2013 Through 21-10-2013",

url = "http://mirror.aclweb.org/emnlp2013/",

}

Nguyen, D-P & Dogruoz, AS 2013, Word level language identification in online multilingual communication. in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (ACL), Michigan, USA, pp. 857-862, Conference on Empirical Methods in Natural Language Processing 2013, Seattle, Washington, United States, 18/10/13. <http://aclweb.org/anthology//D/D13/D13-1084.pdf>

Word level language identification in online multilingual communication. / Nguyen, Dong-Phuong; Dogruoz, A. Seza.
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Michigan, USA: Association for Computational Linguistics (ACL), 2013. p. 857-862.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Word level language identification in online multilingual communication

AU - Nguyen, Dong-Phuong

AU - Dogruoz, A. Seza

PY - 2013/10/18

Y1 - 2013/10/18

N2 - Multilingual speakers switch between languages in online and spoken communication. Analyses of large scale multilingual data require automatic language identification at the word level. For our experiments with multilingual online discussions, we first tag the language of individual words using language models and dictionaries. Secondly, we incorporate context to improve the performance. We achieve an accuracy of 98%. Besides word level accuracy, we use two new metrics to evaluate this task.

AB - Multilingual speakers switch between languages in online and spoken communication. Analyses of large scale multilingual data require automatic language identification at the word level. For our experiments with multilingual online discussions, we first tag the language of individual words using language models and dictionaries. Secondly, we incorporate context to improve the performance. We achieve an accuracy of 98%. Besides word level accuracy, we use two new metrics to evaluate this task.

KW - Language identification

KW - multilingual

KW - METIS-302564

KW - EWI-24092

KW - IR-88555

KW - Social Media

M3 - Conference contribution

SN - 978-1-937284-97-8

SP - 857

EP - 862

BT - Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing

PB - Association for Computational Linguistics (ACL)

CY - Michigan, USA

T2 - Conference on Empirical Methods in Natural Language Processing 2013

Y2 - 18 October 2013 through 21 October 2013

ER -