Ricardo Jorge Espírito

Abstract
Several research works have applied Reinforcement Learning (RL) algorithms to solve the Rate Adaptation (RA) problem in Wi-Fi networks. The dynamic nature of the radio link requires the algorithms to be responsive to changes in link quality. Delays in the execution of the algorithm due to implementional details may be detrimental to its performance, which in turn may decrease network performance. These delays can be avoided to a certain extent. However, this aspect has been overlooked in the state of the art when using simulated environments, since the computational delays are not considered. In this paper, we present an analysis of computational delays and their impact on the performance of RL-based RA algorithms, and propose a methodology to incorporate the experimental computational delays of the algorithms from running in a specific target hardware, in a simulation environment. Our simulation results considering the real computational delays showed that these delays do, in fact, degrade the algorithm’s execution and training capabilities which, in the end, has a negative impact on network performance. © ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2024.