COUNSEL: Cloud Resource Configuration Management using Deep Reinforcement Learning

Internet Clouds are essentially service factories that offer various networked services through different service models, viz., Infrastructure, Platform, Software, and Functions as a Service. Meeting the desired service level objectives (SLOs) while ensuring efficient resource utilization requires significant efforts to provision the associated cloud resources correctly and on time. Therefore, one of the critical issues for any cloud service provider is resource configuration management. On one end, i.e., from the cloud operator's perspective, resource management affects overall resource utilization and efficiency. In contrast, from the cloud user/customer perspective, resource configuration affects the performance, cost, and offered SLOs. However, the state-of-the-art solutions for finding the configurations are limited to a single component or handle static workloads. Further, these solutions are computationally expensive and introduce profiling overhead, limiting scalability. Therefore, we propose COUNSEL, a deep reinforcement learning-based framework to handle the dynamic workloads and efficiently manage the configurations of an arbitrary multi-component service. We evaluate COUNSEL with three initial policies: over-provisioning, under-provisioning, and expert provisioning. In all the cases, COUNSEL eliminates the profiling overhead and achieves the average reward between 20 - 60% without violating the SLOs and budget constraints. Moreover, the inference time of COUNSEL has a constant time complexity.