Empirical co-occurrence rate networks for sequence labeling

Zhemin Zhu, Djoerd Hiemstra, Peter M.G. Apers, Andreas Wombacher

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

48 Downloads (Pure)

Abstract

Sequence labeling has wide applications in many areas. For example, most of named entity recog- nition tasks, which extract named entities or events from unstructured data, can be formalized as sequence labeling problems. Sequence labeling has been studied extensively in different commu- nities, such as data mining, natural language processing or machine learning. Many powerful and popular models have been developed, such as hidden Markov models (HMMs) [4], conditional Markov models (CMMs) [3], and conditional random fields (CRFs) [2]. Despite their successes, they suffer from some known problems: (i) HMMs are generative models which suffer from the mismatch problem, and also it is difficult to incorporate overlapping, non-independent features into a HMM explicitly. (ii) CMMs suffer from the label bias problem; (iii) CRFs overcome the problems of HMMs and CMMs, but the global normalization of CRFs can be very expensive. This prevents CRFs from being applied to big datasets (e.g. Tweets). In this paper, we propose the empirical Co-occurrence Rate Networks (ECRNs) [5] for sequence la- beling. CRNs avoid the problems of the existing models mentioned above. To make the training of CRNs as efficient as possible, we simply use the empirical distribution as the parameter estimation. This results in the ECRNs which can be trained orders of magnitude faster and still obtain compet- itive accuracy to the existing models. ECRN has been applied as a component to the University of Twente system [1] for concept extraction challenge at #MSM2013, which won the best challenge submission awards. ECRNs can be very useful for practitioners on big data.
Original languageUndefined
Title of host publicationDutch-Belgian Database Day, DBDBD 2013
Place of PublicationRotterdam, The Netherlands
PublisherErasmus University Rotterdam
Pages10
Number of pages1
ISBN (Print)not assigned
Publication statusPublished - 29 Nov 2013

Publication series

Name
PublisherErasmus University Rotterdam

Keywords

  • EWI-24073
  • METIS-302560
  • IR-88490
  • DB-DM: DATA MINING

Cite this

Zhu, Z., Hiemstra, D., Apers, P. M. G., & Wombacher, A. (2013). Empirical co-occurrence rate networks for sequence labeling. In Dutch-Belgian Database Day, DBDBD 2013 (pp. 10). Rotterdam, The Netherlands: Erasmus University Rotterdam.
Zhu, Zhemin ; Hiemstra, Djoerd ; Apers, Peter M.G. ; Wombacher, Andreas. / Empirical co-occurrence rate networks for sequence labeling. Dutch-Belgian Database Day, DBDBD 2013. Rotterdam, The Netherlands : Erasmus University Rotterdam, 2013. pp. 10
@inproceedings{fcb7ef85229f483fbfbda762bc07b03a,
title = "Empirical co-occurrence rate networks for sequence labeling",
abstract = "Sequence labeling has wide applications in many areas. For example, most of named entity recog- nition tasks, which extract named entities or events from unstructured data, can be formalized as sequence labeling problems. Sequence labeling has been studied extensively in different commu- nities, such as data mining, natural language processing or machine learning. Many powerful and popular models have been developed, such as hidden Markov models (HMMs) [4], conditional Markov models (CMMs) [3], and conditional random fields (CRFs) [2]. Despite their successes, they suffer from some known problems: (i) HMMs are generative models which suffer from the mismatch problem, and also it is difficult to incorporate overlapping, non-independent features into a HMM explicitly. (ii) CMMs suffer from the label bias problem; (iii) CRFs overcome the problems of HMMs and CMMs, but the global normalization of CRFs can be very expensive. This prevents CRFs from being applied to big datasets (e.g. Tweets). In this paper, we propose the empirical Co-occurrence Rate Networks (ECRNs) [5] for sequence la- beling. CRNs avoid the problems of the existing models mentioned above. To make the training of CRNs as efficient as possible, we simply use the empirical distribution as the parameter estimation. This results in the ECRNs which can be trained orders of magnitude faster and still obtain compet- itive accuracy to the existing models. ECRN has been applied as a component to the University of Twente system [1] for concept extraction challenge at #MSM2013, which won the best challenge submission awards. ECRNs can be very useful for practitioners on big data.",
keywords = "EWI-24073, METIS-302560, IR-88490, DB-DM: DATA MINING",
author = "Zhemin Zhu and Djoerd Hiemstra and Apers, {Peter M.G.} and Andreas Wombacher",
note = "eemcs-eprint-24073",
year = "2013",
month = "11",
day = "29",
language = "Undefined",
isbn = "not assigned",
publisher = "Erasmus University Rotterdam",
pages = "10",
booktitle = "Dutch-Belgian Database Day, DBDBD 2013",

}

Zhu, Z, Hiemstra, D, Apers, PMG & Wombacher, A 2013, Empirical co-occurrence rate networks for sequence labeling. in Dutch-Belgian Database Day, DBDBD 2013. Erasmus University Rotterdam, Rotterdam, The Netherlands, pp. 10.

Empirical co-occurrence rate networks for sequence labeling. / Zhu, Zhemin; Hiemstra, Djoerd; Apers, Peter M.G.; Wombacher, Andreas.

Dutch-Belgian Database Day, DBDBD 2013. Rotterdam, The Netherlands : Erasmus University Rotterdam, 2013. p. 10.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

TY - GEN

T1 - Empirical co-occurrence rate networks for sequence labeling

AU - Zhu, Zhemin

AU - Hiemstra, Djoerd

AU - Apers, Peter M.G.

AU - Wombacher, Andreas

N1 - eemcs-eprint-24073

PY - 2013/11/29

Y1 - 2013/11/29

N2 - Sequence labeling has wide applications in many areas. For example, most of named entity recog- nition tasks, which extract named entities or events from unstructured data, can be formalized as sequence labeling problems. Sequence labeling has been studied extensively in different commu- nities, such as data mining, natural language processing or machine learning. Many powerful and popular models have been developed, such as hidden Markov models (HMMs) [4], conditional Markov models (CMMs) [3], and conditional random fields (CRFs) [2]. Despite their successes, they suffer from some known problems: (i) HMMs are generative models which suffer from the mismatch problem, and also it is difficult to incorporate overlapping, non-independent features into a HMM explicitly. (ii) CMMs suffer from the label bias problem; (iii) CRFs overcome the problems of HMMs and CMMs, but the global normalization of CRFs can be very expensive. This prevents CRFs from being applied to big datasets (e.g. Tweets). In this paper, we propose the empirical Co-occurrence Rate Networks (ECRNs) [5] for sequence la- beling. CRNs avoid the problems of the existing models mentioned above. To make the training of CRNs as efficient as possible, we simply use the empirical distribution as the parameter estimation. This results in the ECRNs which can be trained orders of magnitude faster and still obtain compet- itive accuracy to the existing models. ECRN has been applied as a component to the University of Twente system [1] for concept extraction challenge at #MSM2013, which won the best challenge submission awards. ECRNs can be very useful for practitioners on big data.

AB - Sequence labeling has wide applications in many areas. For example, most of named entity recog- nition tasks, which extract named entities or events from unstructured data, can be formalized as sequence labeling problems. Sequence labeling has been studied extensively in different commu- nities, such as data mining, natural language processing or machine learning. Many powerful and popular models have been developed, such as hidden Markov models (HMMs) [4], conditional Markov models (CMMs) [3], and conditional random fields (CRFs) [2]. Despite their successes, they suffer from some known problems: (i) HMMs are generative models which suffer from the mismatch problem, and also it is difficult to incorporate overlapping, non-independent features into a HMM explicitly. (ii) CMMs suffer from the label bias problem; (iii) CRFs overcome the problems of HMMs and CMMs, but the global normalization of CRFs can be very expensive. This prevents CRFs from being applied to big datasets (e.g. Tweets). In this paper, we propose the empirical Co-occurrence Rate Networks (ECRNs) [5] for sequence la- beling. CRNs avoid the problems of the existing models mentioned above. To make the training of CRNs as efficient as possible, we simply use the empirical distribution as the parameter estimation. This results in the ECRNs which can be trained orders of magnitude faster and still obtain compet- itive accuracy to the existing models. ECRN has been applied as a component to the University of Twente system [1] for concept extraction challenge at #MSM2013, which won the best challenge submission awards. ECRNs can be very useful for practitioners on big data.

KW - EWI-24073

KW - METIS-302560

KW - IR-88490

KW - DB-DM: DATA MINING

M3 - Conference contribution

SN - not assigned

SP - 10

BT - Dutch-Belgian Database Day, DBDBD 2013

PB - Erasmus University Rotterdam

CY - Rotterdam, The Netherlands

ER -

Zhu Z, Hiemstra D, Apers PMG, Wombacher A. Empirical co-occurrence rate networks for sequence labeling. In Dutch-Belgian Database Day, DBDBD 2013. Rotterdam, The Netherlands: Erasmus University Rotterdam. 2013. p. 10