Análise comparativa de algoritmos de aprendizado por reforço: ppo vs sac na plataforma aws deepracer

Abstract

The transfer of Reinforcement Learning (RL) models from simulation to the real world (Sim-to- Real) is a critical challenge in autonomous navigation due to geometric overfitting. This article presents an empirical comparative analysis between the Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) algorithms on the AWS DeepRacer platform. To evaluate generalization robustness, a LiDAR sensor was integrated into the virtual architecture, subjecting the models to 34 hours of training under a continuous reward function (Centerline). The empirical results reveal that, although SAC exhibits high initial sampling efficiency, the algorithm suffers severe stagnation with increasing spatial data dimensionality. In contrast, the conservative update of PPO perfectly assimilated LiDAR, ensuring stability in asymmetrical tracings. It is concluded that PPO presents superior architectural resilience for multisensor integration in autonomous systems