<i>Sakkara</i>: Intelligent Topology-Aware Scheduling for Kubernetes in the Age of AI

Abstract

The rapid growth of Artificial Intelligence (AI) workloads has introduced unprecedented challenges to modern cloud-native systems, particularly in Kubernetes (K8s)-based environments. These workloads often demand low-latency communication, high resource locality, and efficient utilization of heterogeneous hardware devices such as Graphics Processing Units (GPUs) and specialized accelerators. However, the existing scheduling mechanisms in K8s are typically unaware of the underlying physical topology, leading to performance degradation and inefficient resource usage. This paper presents Sakkara, a novel topology-aware scheduling framework designed to optimize the placement of AI workloads in K8s clusters. Sakkara incorporates a hierarchical model of the Data Center (DC), including nodes and racks, enabling flexible scheduling strategies that account for resource availability and risk-aware metrics that mitigate performance interference and constraint violations caused by topology-unaware placement. Sakkara extends existing scheduling logic in K8s with placement strategies that guide pod allocation using configurable topology constraints, aiming to minimize communication costs and maximize workload performance. We evaluated Sakkara on a representative AI workload, a distributed training application under different cluster configurations. Experimental results show that Sakkara improves job completion time, throughput, and memory utilization compared to available K8s schedulers, achieving improvements of up to 10%. Sakkara, available as open-source, offers a promising pathway toward topology-conscious orchestration of AI workloads in next-generation cloud environments.