Towards continuous control for mobile robot navigation: A reinforcement learning and slam based approach

    Research output: Contribution to journalConference articleAcademicpeer-review

    27 Downloads (Pure)

    Abstract

    We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

    Original languageEnglish
    Pages (from-to)857-863
    Number of pages7
    JournalInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
    Volume42
    Issue number2/W13
    DOIs
    Publication statusPublished - 4 Jun 2019
    Event4th ISPRS Geospatial Week 2019 - University of Twente, Enschede, Netherlands
    Duration: 10 Jun 201914 Jun 2019
    Conference number: 4
    https://www.gsw2019.org/

    Fingerprint

    Reinforcement learning
    robot
    reinforcement
    Mobile robots
    navigation
    Navigation
    learning
    reward
    collision
    Angular velocity
    Motion planning
    Robots
    sensor
    filter
    planning
    experiment
    Sensors
    evaluation
    Experiments

    Keywords

    • Artificial intelligence
    • Deep reinforcement learning
    • Online path planning
    • SLAM
    • Unknown environments

    Cite this

    @article{09d520ee3ed14ab38a07d82c31659bfe,
    title = "Towards continuous control for mobile robot navigation: A reinforcement learning and slam based approach",
    abstract = "We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83{\%} compared to 56{\%} in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.",
    keywords = "Artificial intelligence, Deep reinforcement learning, Online path planning, SLAM, Unknown environments",
    author = "K.A.A. Mustafa and N. Botteghi and B. Sirmacek and M. Poel and S. Stramigioli",
    year = "2019",
    month = "6",
    day = "4",
    doi = "10.5194/isprs-archives-XLII-2-W13-857-2019",
    language = "English",
    volume = "42",
    pages = "857--863",
    journal = "International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences",
    issn = "1682-1750",
    publisher = "Copernicus",
    number = "2/W13",

    }

    TY - JOUR

    T1 - Towards continuous control for mobile robot navigation

    T2 - A reinforcement learning and slam based approach

    AU - Mustafa, K.A.A.

    AU - Botteghi, N.

    AU - Sirmacek, B.

    AU - Poel, M.

    AU - Stramigioli, S.

    PY - 2019/6/4

    Y1 - 2019/6/4

    N2 - We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

    AB - We introduce a new autonomous path planning algorithm for mobile robots for reaching target locations in an unknown environment where the robot relies on its on-board sensors. In particular, we describe the design and evaluation of a deep reinforcement learning motion planner with continuous linear and angular velocities to navigate to a desired target location based on deep deterministic policy gradient (DDPG). Additionally, the algorithm is enhanced by making use of the available knowledge of the environment provided by a grid-based SLAM with Rao-Blackwellized particle filter algorithm in order to shape the reward function in an attempt to improve the convergence rate, escape local optima and reduce the number of collisions with the obstacles. A comparison is made between a reward function shaped based on the map provided by the SLAM algorithm and a reward function when no knowledge of the map is available. Results show that the required learning time has been decreased in terms of number of episodes required to converge, which is 560 episodes compared to 1450 episodes in the standard RL algorithm, after adopting the proposed approach and the number of obstacle collision is reduced as well with a success ratio of 83% compared to 56% in the standard RL algorithm. The results are validated in a simulated experiment on a skid-steering mobile robot.

    KW - Artificial intelligence

    KW - Deep reinforcement learning

    KW - Online path planning

    KW - SLAM

    KW - Unknown environments

    UR - http://www.scopus.com/inward/record.url?scp=85067504995&partnerID=8YFLogxK

    U2 - 10.5194/isprs-archives-XLII-2-W13-857-2019

    DO - 10.5194/isprs-archives-XLII-2-W13-857-2019

    M3 - Conference article

    AN - SCOPUS:85067504995

    VL - 42

    SP - 857

    EP - 863

    JO - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

    JF - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences

    SN - 1682-1750

    IS - 2/W13

    ER -