Diktyo: Network-Aware Scheduling in Container-Based Clouds

Abstract

Containers have revolutionized application deployment and life-cycle management in current cloud platforms. Applications have evolved from single monoliths to complex graphs of loosely-coupled microservices. However, the efficient allocation of microservice-based applications is challenging due to their complex inter-dependencies. Further, recent applications are becoming even more delay-sensitive, demanding lower latency between dependent microservices. Scheduling policies in popular container orchestration platforms mainly aim to increase the resource efficiency of the infrastructure, insufficient for latency-sensitive applications. Application domains such as the Internet of Things and multi-tier Web services would benefit from network-aware policies that consider network latency and bandwidth in the scheduling process. Previous works have studied network-aware scheduling via theoretical formulations or heuristic-based methods evaluated via simulations or small testbeds, making their full applicability in popular platforms difficult. This paper proposes a novel network-aware framework for the popular Kubernetes (K8s) platform named Diktyo that determines the placement of dependent microservices in long-running applications focused on reducing the application's end-to-end latency and guaranteeing bandwidth reservations. Simulations show that Diktyo can significantly reduce the network latency for various applications across different infrastructure topologies compared to default K8s scheduling plugins. Also, experiments in a K8s cluster with microservice benchmark applications show that Diktyo can increase database throughput by 22% and reduce application response time by 45%.