DipDCE: Offloading-Aware Vision Inference in Edge with Concurrent Executions

Abstract

Distributed computer vision deployment frameworks aim to reduce end-to-end latency and backbone traffic by distributing computation across edge and cloud tiers. In typical architectures, nodes can process a portion of incoming image streams locally or in an edge server and offload the remainder to the cloud. However, runtime decisions about task scheduling and offloading often impose a significant computational burden on nodes, leading to unnecessary runtime overhead and underutilization of hardware resources for deployed applications. To address these challenges, we propose an offline mixed-integer optimisation framework that partitions workloads into local edge processing and deterministic offloading to the cloud tier. Each edge node handles multiple image streams by deploying parallel processing pipelines. Our framework enforces quality-of-service (QoS) constraints while maximising the proportion of computation performed at the edge. By precomputing an optimal configuration, the edge node can meet latency and resource requirements without incurring the overhead of on-the-fly optimisation. We evaluate our approach against several standard image stream processing approaches and demonstrate that our method satisfies QoS targets with lower processing delay and higher resource utilisation. We demonstrate that as the number of sensors increases, our framework can adjust the deployed configuration to enhance on-edge processing.