Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization

Zhemin Zhu, Djoerd Hiemstra, Peter M.G. Apers, Andreas Wombacher

Research output: Book/ReportReportProfessional

12 Downloads (Pure)

Abstract

The standard training method of Conditional Random Fields (CRFs) is very slow for large-scale applications. As an alternative, piecewise training divides the full graph into pieces, trains them independently, and combines the learned weights at test time. In this paper, we present separate training for undirected models based on the novel Co-occurrence Rate Factorization (CR-F). Separate training is a local training method. In contrast to piecewise training, separate training is exact. In contrast to MEMMs, separate training is unaffected by the label bias problem. Experiments show that separate training (i) is unaffected by the label bias problem; (ii) reduces the training time from weeks to seconds; and (iii) obtains competitive results to the standard and piecewise training on linear-chain CRFs.
Original languageUndefined
Place of PublicationEnschede
PublisherCentre for Telematics and Information Technology (CTIT)
Number of pages10
Publication statusPublished - 1 Oct 2012

Publication series

NameCTIT Technical Report Series
PublisherCentre for Telematics and Information Technology, University of Twente
No.TR-CTIT-12-29
ISSN (Print)1381-3625

Keywords

  • Conditional random fields
  • METIS-296153
  • IR-84371
  • EWI-22600
  • undirected graph factorization
  • natural language processing

Cite this

Zhu, Z., Hiemstra, D., Apers, P. M. G., & Wombacher, A. (2012). Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization. (CTIT Technical Report Series; No. TR-CTIT-12-29). Enschede: Centre for Telematics and Information Technology (CTIT).
Zhu, Zhemin ; Hiemstra, Djoerd ; Apers, Peter M.G. ; Wombacher, Andreas. / Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization. Enschede : Centre for Telematics and Information Technology (CTIT), 2012. 10 p. (CTIT Technical Report Series; TR-CTIT-12-29).
@book{721637a386e24af6a7182a4f9165a9de,
title = "Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization",
abstract = "The standard training method of Conditional Random Fields (CRFs) is very slow for large-scale applications. As an alternative, piecewise training divides the full graph into pieces, trains them independently, and combines the learned weights at test time. In this paper, we present separate training for undirected models based on the novel Co-occurrence Rate Factorization (CR-F). Separate training is a local training method. In contrast to piecewise training, separate training is exact. In contrast to MEMMs, separate training is unaffected by the label bias problem. Experiments show that separate training (i) is unaffected by the label bias problem; (ii) reduces the training time from weeks to seconds; and (iii) obtains competitive results to the standard and piecewise training on linear-chain CRFs.",
keywords = "Conditional random fields, METIS-296153, IR-84371, EWI-22600, undirected graph factorization, natural language processing",
author = "Zhemin Zhu and Djoerd Hiemstra and Apers, {Peter M.G.} and Andreas Wombacher",
year = "2012",
month = "10",
day = "1",
language = "Undefined",
series = "CTIT Technical Report Series",
publisher = "Centre for Telematics and Information Technology (CTIT)",
number = "TR-CTIT-12-29",
address = "Netherlands",

}

Zhu, Z, Hiemstra, D, Apers, PMG & Wombacher, A 2012, Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization. CTIT Technical Report Series, no. TR-CTIT-12-29, Centre for Telematics and Information Technology (CTIT), Enschede.

Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization. / Zhu, Zhemin; Hiemstra, Djoerd; Apers, Peter M.G.; Wombacher, Andreas.

Enschede : Centre for Telematics and Information Technology (CTIT), 2012. 10 p. (CTIT Technical Report Series; No. TR-CTIT-12-29).

Research output: Book/ReportReportProfessional

TY - BOOK

T1 - Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization

AU - Zhu, Zhemin

AU - Hiemstra, Djoerd

AU - Apers, Peter M.G.

AU - Wombacher, Andreas

PY - 2012/10/1

Y1 - 2012/10/1

N2 - The standard training method of Conditional Random Fields (CRFs) is very slow for large-scale applications. As an alternative, piecewise training divides the full graph into pieces, trains them independently, and combines the learned weights at test time. In this paper, we present separate training for undirected models based on the novel Co-occurrence Rate Factorization (CR-F). Separate training is a local training method. In contrast to piecewise training, separate training is exact. In contrast to MEMMs, separate training is unaffected by the label bias problem. Experiments show that separate training (i) is unaffected by the label bias problem; (ii) reduces the training time from weeks to seconds; and (iii) obtains competitive results to the standard and piecewise training on linear-chain CRFs.

AB - The standard training method of Conditional Random Fields (CRFs) is very slow for large-scale applications. As an alternative, piecewise training divides the full graph into pieces, trains them independently, and combines the learned weights at test time. In this paper, we present separate training for undirected models based on the novel Co-occurrence Rate Factorization (CR-F). Separate training is a local training method. In contrast to piecewise training, separate training is exact. In contrast to MEMMs, separate training is unaffected by the label bias problem. Experiments show that separate training (i) is unaffected by the label bias problem; (ii) reduces the training time from weeks to seconds; and (iii) obtains competitive results to the standard and piecewise training on linear-chain CRFs.

KW - Conditional random fields

KW - METIS-296153

KW - IR-84371

KW - EWI-22600

KW - undirected graph factorization

KW - natural language processing

M3 - Report

T3 - CTIT Technical Report Series

BT - Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization

PB - Centre for Telematics and Information Technology (CTIT)

CY - Enschede

ER -

Zhu Z, Hiemstra D, Apers PMG, Wombacher A. Separate Training for Conditional Random Fields Using Co-occurrence Rate Factorization. Enschede: Centre for Telematics and Information Technology (CTIT), 2012. 10 p. (CTIT Technical Report Series; TR-CTIT-12-29).