Quantifying the Impact of Job Placement and Routing on Network Efficiency in AI Clusters

Abstract

High-performance computing (HPC) clusters are essential for training large-scale AI models, yet they often suffer from severe underutilization due to network bottlenecks. This paper investigates the critical role of job placement in multi-tenant AI clusters and its impact on network performance. We propose a flow-level system model that jointly considers job placement, network topology, and routing strategy to evaluate link loads and congestion. By analyzing optimal, random and state-of-the-art placement strategies across modern network topologies, we demonstrate that placement decisions significantly influence network efficiency. Our results show that job placement cannot be ignored even under optimal routing, and that existing placement strategies are dependent on the routing strategy. This work underscores the importance of prioritizing job placement, as suboptimal placements can lead to significant performance degradation in AI workloads on HPC infrastructure.