Flexible and Efficient Feature-Level Fusion With Wireless Acoustic Sensors Using Graph Attention Networks

Abstract

Wireless acoustic sensor networks (WASNs), or the Internet of Audio Things (IoAuT), enable intelligent acoustic sensing in IoT applications such as smart homes. A key challenge in such deployments is achieving accurate results under limited bandwidth and energy constraints. To efficiently process audio, sensor fusion techniques can be used. They can aggregate the raw audio signals, features, or local decisions of all sensors to make the final decision for tasks such as acoustic event classification. In contrast to a wired setting, connections within WASNs may be unstable due to interference. Additionally, transmitting large amounts of data reduces the sensors’ battery lifespan. A data/signal-level fusion preserves the full information by transmitting and fusing the raw audio, but it imposes an impractical bandwidth burden for WASNs. Conversely, decision-level fusion is communication-efficient and supports a variable number of sensors; it may compromise accuracy. In contrast, existing feature-level fusion methods can transmit richer information to the fusion center, but incur a higher communication overhead and often necessitate a fixed sensor topology, rendering them less suitable for wireless IoT settings. In this work, we propose a new feature-level fusion framework based on graph attention networks (GATs) for acoustic event classification tasks using a WASN. Our approach supports dynamic WASN topologies and introduces a message condensation layer that reduces the volume of transmitted data, lowering the communication cost and bandwidth usage. Empirical results of a domestic acoustic event classification task show that our framework outperforms decision-level fusion techniques while maintaining a similar communication cost. Moreover, our framework outperforms prior feature-level and data-level fusion methods with a notably reduced communication cost by reducing the size of transmitted messages, and the additional flexibility of supporting dynamic WASN topologies, thus being robust to sensor failures, or the addition or removal of sensors.