Distributed Multi‐Agent RL‐Based Autonomous Spectrum Allocation in D2D‐Enabled Multi‐Tier HetNets

This chapter proposes a distributive reinforcement learning (RL)-based resource allocation scheme in multi-tier heterogeneous network (HetNet) to mitigate interference between device-to-device (D2D) users and cellular users. The D2D-enabled HetNets can induce significant challenges in terms of co-tier and cross-tier interference, and resource allocation becomes quite challenging. RL has been greatly employed in providing solution to the resource allocation problems in fifth-generation networks. The chapter analyzes the effects of increase in base station density and network tier over the performance of D2D users. It considers a network model consists of a multi-tier HetNet with macro, micro, pico, and femto base stations and D2D users who operate in underlay mode. Multiple D2D users are uniformly distributed in the network and reuse the uplink cellular resources. The learning methodology employed makes use of interactions with the environment to adapt to changing network conditions in fast and convenient manner to improve the throughput and spectral efficiency.