Factorizing Probabilistic Graphical Models Using Co-occurrence Rate

Factorization is of fundamental importance in the area of Probabilistic Graphical Models (PGMs). In this paper, we theoretically develop a novel mathematical concept, \textbf{C}o-occurrence \textbf{R}ate (CR), for factorizing PGMs. CR has three obvious advantages: (1) CR provides a unified mathematical foundation for factorizing different types of PGMs. We show that Bayesian Network Factorization (BN-F), Conditional Random Field Factorization (CRF-F), Markov Random Field Factorization (MRF-F) and Refined Markov Random Field Factorization (RMRF-F) are all special cases of CR Factorization (CR-F); (2) CR has simple probability definition and clear intuitive interpretation. CR-F tells not only the scopes of the factors, but also the exact probability functions of these factors; (3) CR connects probability factorization and graph operations perfectly. The factorization process of CR-F can be visualized as applying a sequence of graph operations including partition, merge, duplicate and condition to a PGM graph. We further obtain an important result: by CR-F, on TCG graphs the scopes of factors can be exactly over maximal cliques without any default configuration. This improves the results of (R)MRF-F which need default configurations, and also indicates that (R)MRF-F, as special cases of CR-F, can not always achieve the optimal results of CR-F.