Natural Policy Gradients In Reinforcement Learning Explained

W.J.A. van Heeswijk

doi:10.48550/arXiv.2209.01820

Natural Policy Gradients In Reinforcement Learning Explained

W.J.A. van Heeswijk

Research output: Working paper › Preprint › Academic

1 Downloads (Pure)

Abstract

Traditional policy gradient methods are fundamentally flawed. Natural gradients converge quicker and better, forming the foundation of contemporary Reinforcement Learning such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). This lecture note aims to clarify the intuition behind natural policy gradients, focusing on the thought process and the key mathematical constructs.

Original language	English
Publisher	ArXiv.org
Number of pages	15
DOIs	https://doi.org/10.48550/arXiv.2209.01820
Publication status	Published - 5 Sept 2022

Keywords

cs.LG
math.OC

Access to Document

10.48550/arXiv.2209.01820

2209.01820v1Final published version, 361 KB

Cite this

@techreport{44b41289b027432c9e4a9861d3455cd1,

title = "Natural Policy Gradients In Reinforcement Learning Explained",

abstract = "Traditional policy gradient methods are fundamentally flawed. Natural gradients converge quicker and better, forming the foundation of contemporary Reinforcement Learning such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). This lecture note aims to clarify the intuition behind natural policy gradients, focusing on the thought process and the key mathematical constructs.",

keywords = "cs.LG, math.OC",

author = "\{van Heeswijk\}, W.J.A.",

note = "14 pages, 3 figures",

year = "2022",

month = sep,

day = "5",

doi = "10.48550/arXiv.2209.01820",

language = "English",

publisher = "ArXiv.org",

type = "WorkingPaper",

institution = "ArXiv.org",

}

TY - UNPB

T1 - Natural Policy Gradients In Reinforcement Learning Explained

AU - van Heeswijk, W.J.A.

N1 - 14 pages, 3 figures

PY - 2022/9/5

Y1 - 2022/9/5

N2 - Traditional policy gradient methods are fundamentally flawed. Natural gradients converge quicker and better, forming the foundation of contemporary Reinforcement Learning such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). This lecture note aims to clarify the intuition behind natural policy gradients, focusing on the thought process and the key mathematical constructs.

AB - Traditional policy gradient methods are fundamentally flawed. Natural gradients converge quicker and better, forming the foundation of contemporary Reinforcement Learning such as Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). This lecture note aims to clarify the intuition behind natural policy gradients, focusing on the thought process and the key mathematical constructs.

KW - cs.LG

KW - math.OC

U2 - 10.48550/arXiv.2209.01820

DO - 10.48550/arXiv.2209.01820

M3 - Preprint

BT - Natural Policy Gradients In Reinforcement Learning Explained

PB - ArXiv.org

ER -

Natural Policy Gradients In Reinforcement Learning Explained

Abstract

Keywords

Access to Document

Fingerprint

Cite this