Closed form maximum likelihood estimator of conditional random fields

Zhemin Zhu, Djoerd Hiemstra, Peter M.G. Apers, Andreas Wombacher

Research output: Book/ReportReportProfessional

119 Downloads (Pure)


Training Conditional Random Fields (CRFs) can be very slow for big data. In this paper, we present a new training method for CRFs called {\em Empirical Training} which is motivated by the concept of co-occurrence rate. We show that the standard training (unregularized) can have many maximum likelihood estimations (MLEs). Empirical training has a unique closed form MLE which is also a MLE of the standard training. We are the first to identify the \emph{Test Time Problem} of the standard training which may lead to low accuracy. Empirical training is immune to this problem. Empirical training is also unaffected by the label bias problem even it is locally normalized. All of these have been verified by experiments. Experiments also show that empirical training reduces the training time from weeks to seconds, and obtains competitive results to the standard and piecewise training on linear-chain CRFs, especially when data are insufficient.
Original languageUndefined
Place of PublicationEnschede
PublisherCentre for Telematics and Information Technology (CTIT)
Publication statusPublished - 12 Feb 2013

Publication series

NameCTIT Technical Report Series
PublisherCentre for Telematics and Information Technology, University of Twente
ISSN (Print)1381-3625


  • EWI-23097
  • IR-84386
  • METIS-296313

Cite this