Análise de desempenho e generalização do algoritmo Soft Actor-Critic (SAC) integrado à otimização de trajetória K1999 em veículos autônomos simulados

Abstract

The development of simulated autonomous vehicles (AWS DeepRacer) faces the challenge of convergence instability and the high exploratory cost inherent to the Soft Actor-Critic (SAC) algorithm in continuous action spaces. This work evaluates the impact of integrating geometric trajectory optimization (K1999) as a structured guide for SAC exploration, aiming to mitigate initial exploratory chaos and geometric overfitting. The AWS cloud environment was used to train two SAC models for 6 hours each: a control (centerline) and a proposed model (K1999). The reward func- tion incorporated exponential distance and velocity decay to the ideal line. Results show that the K1999 model mitigated exploratory chaos, consolidating its optimal policy 74 iterations earlier than the baseline. In zero-shot generalization tests on an unseen track, the approach reduced collisions by approximately 66% and cut the lap time in half. In conclusion, injecting pre-calculated heuristics reduces unproductive entropy and prevents catastrophic failures of SAC, although the model still exhibits residual rates of overfitting when dealing with entirely unknown scenarios.