gym-hpa: Efficient Auto-Scaling via Reinforcement Learning for Complex Microservice-based Applications in Kubernetes

Abstract

Containers have revolutionized application deployment and life-cycle management in current cloud platforms. Applications have evolved from large monoliths to complex graphs of loosely-coupled microservices aiming to improve deployment flexibility and operational efficiency. However, modern microservice-based architectures are challenging since proper allocation and scaling of microservices is a difficult task due to their complex inter-dependencies. Existing works do not consider microservice dependencies, which could lead to the application’s performance degradation when service demand increases. This paper studies the impact of microservice interdependencies in auto-scaling mechanisms by proposing a novel framework named gym-hpa that enables different auto-scaling goals via Reinforcement Learning (RL). The framework has been developed based on the OpenAI Gym library for the popular Kubernetes (K8s) platform to bridge the gap between RL and auto-scaling research by training RL agents on real cloud environments. The aim is to improve resource usage and reduce the application’s response time in future cloud platforms by considering microservice inter-dependencies in horizontal scaling. Experiments with microservice benchmark applications show that RL agents trained with the gym-hpa framework can reduce on average resource usage by 30% and reduce the application’s response time by 25% compared to default scaling mechanisms.