Towards continuous control for mobile robot navigation: A reinforcement learning and slam based approach

Research output: Contribution to journalConference articleAcademicpeer-review

23 Downloads (Pure)

Abstract

We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

Original languageEnglish
Pages (from-to)857-863
Number of pages7
JournalInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
Volume42
Issue number2/W13
DOIs
Publication statusPublished - 4 Jun 2019
Event4th ISPRS Geospatial Week 2019 - University of Twente, Enschede, Netherlands
Duration: 10 Jun 201914 Jun 2019
Conference number: 4
https://www.gsw2019.org/

Fingerprint

Reinforcement learning
robot
reinforcement
Mobile robots
navigation
Navigation
learning
reward
collision
Angular velocity
Motion planning
Robots
sensor
filter
planning
experiment
Sensors
evaluation
Experiments

Keywords

  • Artificial intelligence
  • Deep reinforcement learning
  • Online path planning
  • SLAM
  • Unknown environments

Cite this

@article{09d520ee3ed14ab38a07d82c31659bfe,
title = "Towards continuous control for mobile robot navigation: A reinforcement learning and slam based approach",
abstract = "We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83{\%} compared to 56{\%} in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.",
keywords = "Artificial intelligence, Deep reinforcement learning, Online path planning, SLAM, Unknown environments",
author = "K.A.A. Mustafa and N. Botteghi and B. Sirmacek and M. Poel and S. Stramigioli",
year = "2019",
month = "6",
day = "4",
doi = "10.5194/isprs-archives-XLII-2-W13-857-2019",
language = "English",
volume = "42",
pages = "857--863",
journal = "International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
issn = "1682-1750",
publisher = "Copernicus",
number = "2/W13",

}

TY - JOUR

T1 - Towards continuous control for mobile robot navigation

T2 - A reinforcement learning and slam based approach

AU - Mustafa, K.A.A.

AU - Botteghi, N.

AU - Sirmacek, B.

AU - Poel, M.

AU - Stramigioli, S.

PY - 2019/6/4

Y1 - 2019/6/4

N2 - We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

AB - We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

KW - Artificial intelligence

KW - Deep reinforcement learning

KW - Online path planning

KW - SLAM

KW - Unknown environments

UR - http://www.scopus.com/inward/record.url?scp=85067504995&partnerID=8YFLogxK

U2 - 10.5194/isprs-archives-XLII-2-W13-857-2019

DO - 10.5194/isprs-archives-XLII-2-W13-857-2019

M3 - Conference article

VL - 42

SP - 857

EP - 863

JO - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

JF - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

SN - 1682-1750

IS - 2/W13

ER -