Q-learning Assisted Energy-Aware Traffic Offloading and Cell Switching in Heterogeneous Networks

Cell switching has been identified as a major approach to significantly reduce the energy consumption of Heterogeneous Networks (HetNets). The main idea behind cell switching is to turn off idle or lightly loaded Base Stations (BSs) and to offload their traffic to neighbouring active cell(s). However, the impact of the offloaded traffic on the power consumption of the neighbouring cell(s) has not been studied sufficiently in the literature, thereby leading to the development of sub-optimal cell switching mechanisms. In this work, we first considered a Control/Data Separated Architecture (CDSA) with a macro cell serving as the Control Base Station (CBS) and multiple small cells as Data Base Stations (DBS). Then, a Q-learning assisted cell switching algorithm is developed in order to determine the small cells to switch off by considering the increase in power consumption of the macro cell due to offloaded traffic from the sleeping cells. The capacity of the macro cell is also taken into consideration to ensure that the Quality of Service (QoS) requirements of users are maintained. Simulation results show that the proposed cell switching algorithm can achieve up to 50% reduction in the total energy consumption of the considered HetNet scenario.