Abstract
Problems in operations research are typically combinatorial and high-dimensional. To a degree, linear programs may efficiently solve such large decision problems. For stochastic multi-period problems, decomposition into a sequence of one-stage decisions with approximated downstream effects is often necessary, e.g., by deploying reinforcement learning to obtain value function approximations (VFAs). When embedding such VFAs into one-stage linear programs, VFA design is restricted by linearity. This paper presents an integrated simulation approach for such complex optimization problems, developing a deep reinforcement learning algorithm that combines linear programming and neural network VFAs. Our proposed method embeds neural network VFAs into one-stage linear decision problems, combining the nonlinear expressive power of neural networks with the efficiency of solving linear programs. As a proof of concept, we perform numerical experiments on a transportation problem. The neural network VFAs consistently outperform polynomial VFAs as well as other benchmarks, with limited design and tuning effort.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2020 Winter Simulation Conference, WSC 2020 |
Editors | K.-H. Bae, B. Feng, S. Kim, S. Lazarova-Molnar, Z. Zheng, T. Roeder, R. Thiesing |
Place of Publication | Piscataway, NJ |
Publisher | IEEE |
Pages | 1063-1074 |
Number of pages | 12 |
ISBN (Electronic) | 978-1-7281-9499-8 |
ISBN (Print) | 978-1-7281-9500-1 |
DOIs | |
Publication status | Published - 29 Mar 2021 |
Externally published | Yes |
Event | Winter Simulation Conference, WSC 2020: Simulation Drives Innovation - Virtual Conference, Orlando, United States Duration: 14 Dec 2020 → 18 Dec 2020 http://meetings2.informs.org/wordpress/wsc2020/ |
Publication series
Name | Proceedings - Winter Simulation Conference |
---|---|
Publisher | IEEE |
Volume | 2020 |
ISSN (Print) | 0891-7736 |
ISSN (Electronic) | 1558-4305 |
Conference
Conference | Winter Simulation Conference, WSC 2020 |
---|---|
Abbreviated title | WSC 2020 |
Country/Territory | United States |
City | Orlando |
Period | 14/12/20 → 18/12/20 |
Internet address |
Keywords
- 22/2 OA procedure