### Abstract

We consider the Hamiltonian cycle problem (HCP) embedded in a controlled Markov decision process. In this setting, HCP reduces to an optimization problem on a set of Markov chains corresponding to a given graph. We prove that Hamiltonian cycles are minimizers for the trace of the fundamental matrix on a set of all stochastic transition matrices. In case of doubly stochastic matrices with symmetric linear perturbation, we show that Hamiltonian cycles minimize a diagonal element of a fundamental matrix for all admissible values of the perturbation parameter. In contrast to the previous work on this topic, our arguments are primarily based on probabilistic rather than algebraic methods.

Original language | Undefined |
---|---|

Place of Publication | Enschede |

Publisher | University of Twente, Department of Applied Mathematics |

Number of pages | 12 |

Publication status | Published - Jun 2007 |

### Publication series

Name | Memorandum |
---|---|

Publisher | University of Twente, Department of Applied Mathematics |

No. | 06472/1841 |

ISSN (Print) | 1874-4850 |

ISSN (Electronic) | 1874-4850 |

### Keywords

- EWI-10244
- MSC-05C45
- Singular perturbation
- Hamiltonian cycle
- Fundamental matrix
- MSC-60J10
- Markov chains
- IR-64109
- MSC-15A51
- METIS-241665

## Cite this

Litvak, N., & Ejov, V. (2007).

*Markov chains and optimality of the Hamiltonian cycle*. (Memorandum; No. 06472/1841). Enschede: University of Twente, Department of Applied Mathematics.