Distributed Parallel Hyperspectral Unmixing for Large-Scale Data in Spark Environments via Geometric Distance

Abstract

Hyperspectral unmixing addresses the challenge of mixed pixels in hyperspectral images by identifying the number of pure pixels (endmembers), extracting their spectral signatures, and estimating their proportions (abundances) in each pixel composing the scene. Traditional hyperspectral unmixing methods often struggle with scalability and computational efficiency when dealing with gigabyte-scale datasets. In this article, we propose a distributed parallel geometric distance (DPGD) method for hyperspectral unmixing, exploring the computational power and benefits of distributed parallel processing within a distributed computing framework. The proposed DPGD leverages geometric distance measurements to accurately identify endmembers and estimate their abundances, taking into account the intrinsic similarities within hyperspectral images. This provides a clearer representation of the data structure, leading to improved unmixing accuracy. By using the Spark programming model, the computational workload is efficiently distributed across multiple nodes, significantly reducing processing time. Experimental results on real hyperspectral datasets demonstrate that DPGD scales effectively up to 32 nodes and 290.9 GB of data, achieving competitive accuracy and efficiency compared to state-of-the-art methods. The code is available at https://github.com/ccaadaro/DPDG