A Study on Reinforcement Learning based Control of Quadcopter with a Cable-suspended Payload

Flying a drone is as simple as playing a video game. However, the suspension of the payload underneath complicates its dynamic behavior and makes control challenging. The default onboard control algorithms are not designed to cope with the unknown interaction introduced by the suspended payload, i.e., the payload's oscillations. Attempts have been made previously using model-based adaptive control techniques to solve this problem. Another way of addressing this problem is using data-driven control techniques such as Reinforcement Learning (RL). RL techniques have been proven to perform well in modeling complex, coupled, and unknown dynamics. This work discusses a study of implementing the RL based controller for manual flying of the quadcopter with a cable-suspended payload system. The simulations are carried out in a specially designed physics environment that simulates the dynamical behavior of a quadcopter-payload system. The RL agent is trained using the proximal policy optimization approach, and numerous simulations are run to ensure that performance is as expected. Finally, the process of putting the provided controller into actual hardware is covered along with any potential difficulties.