Empirical co-occurrence rate networks for sequence labeling

Zhemin Zhu, Djoerd Hiemstra, Peter M.G. Apers, Andreas Wombacher

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

3 Citations (Scopus)
21 Downloads (Pure)

Abstract

Structured prediction has wide applications in many areas. Powerful and popular models for structured prediction have been developed. Despite the successes, they suffer from some known problems: (i) Hidden Markov models are generative models which suffer from the mismatch problem. Also it is difficult to incorporate overlapping, non-independent features into a hidden Markov model explicitly. (ii) Conditional Markov models suffer from the label bias problem. (iii) Conditional Random Fields (CRFs) overcome the label bias problem by global normalization. But the global normalization of CRFs can be expensive which prevents CRFs from applying to big data. In this paper, we propose the Empirical Co-occurrence Rate Networks (ECRNs) for sequence labeling. ECRNs are discriminative models, so ECRNs overcome the problems of HMMs. ECRNs are also immune to the label bias problem even though they are locally normalized. To make the estimation of ECRNs as fast as possible, we simply use the empirical distributions as the estimation of parameters. Experiments on two real-world NLP tasks show that ECRNs reduce the training time radically while obtain competitive accuracy to the state-of-the-art models.
Original languageUndefined
Title of host publicationProceedings of the 12th International Conference on Machine Learning and Applications, ICMLA 2013
Place of PublicationUSA
PublisherIEEE Computer Society
Pages93-98
Number of pages6
ISBN (Print)978-0-7695-5144-9
DOIs
Publication statusPublished - Dec 2013
Event12th International Conference on Machine Learning and Applications, ICMLA 2013, Miami Beach, FL, USA: Proceedings of the 12th International Conference on Machine Learning and Applications, ICMLA 2013 - USA
Duration: 1 Dec 2013 → …

Publication series

Name
PublisherIEEE Computer Society

Conference

Conference12th International Conference on Machine Learning and Applications, ICMLA 2013, Miami Beach, FL, USA
CityUSA
Period1/12/13 → …

Keywords

  • EWI-24071
  • METIS-302559
  • IR-88489
  • DB-DM: DATA MINING

Cite this