## Abstract

Problems in operations research are typically combinatorial and high-dimensional. To a degree, linear programs may efficiently solve such large decision problems. For stochastic multi-period problems, decomposition into a sequence of one-stage decisions with approximated downstream effects is often necessary, e.g., by deploying reinforcement learning to obtain value function approximations (VFAs). When embedding such VFAs into one-stage linear programs, VFA design is restricted by linearity. This paper presents an integrated simulation approach for such complex optimization problems, developing a deep reinforcement learning algorithm that combines linear programming and neural network VFAs. Our proposed method embeds neural network VFAs into one-stage linear decision problems, combining the nonlinear expressive power of neural networks with the efficiency of solving linear programs. As a proof of concept, we perform numerical experiments on a transportation problem. The neural network VFAs consistently outperform polynomial VFAs as well as other benchmarks, with limited design and tuning effort.

Original language | English |
---|---|

Title of host publication | Proceedings of the 2020 Winter Simulation Conference, WSC 2020 |

Editors | K.-H. Bae, B. Feng, S. Kim, S. Lazarova-Molnar, Z. Zheng, T. Roeder, R. Thiesing |

Place of Publication | Piscataway, NJ |

Publisher | IEEE |

Pages | 1063-1074 |

Number of pages | 12 |

ISBN (Electronic) | 978-1-7281-9499-8 |

ISBN (Print) | 978-1-7281-9500-1 |

DOIs | |

Publication status | Published - 29 Mar 2021 |

Externally published | Yes |

Event | Winter Simulation Conference, WSC 2020: Simulation Drives Innovation - Virtual Conference, Orlando, United States Duration: 14 Dec 2020 → 18 Dec 2020 http://meetings2.informs.org/wordpress/wsc2020/ |

### Publication series

Name | Proceedings - Winter Simulation Conference |
---|---|

Publisher | IEEE |

Volume | 2020 |

ISSN (Print) | 0891-7736 |

ISSN (Electronic) | 1558-4305 |

### Conference

Conference | Winter Simulation Conference, WSC 2020 |
---|---|

Abbreviated title | WSC 2020 |

Country/Territory | United States |

City | Orlando |

Period | 14/12/20 → 18/12/20 |

Internet address |

## Keywords

- 22/2 OA procedure