FF-GAT: Feature Fusion Using Graph Attention Networks

Abstract

Convolutional neural networks (CNNs) have accomplished magnificent performance on object classification tasks. This work introduces a novel image classification approach based on feature vector fusion of two CNN architectures using graph attention networks (GAT). In the proposed method we extract feature maps from shallow and deep layers of two CNN architectures. These extracted feature vectors are represented as nodes in a graph, and edges between nodes are constructed based on similarities between the feature vectors. The GAT is used to aggregate and fuse connected nodes based on their importance and relevance to the classification task. We believe that this approach compensates for convolution defects during feature processing when using a single CNN. This paper attempts to show that graph-based deep learning can be used to fuse two CNN architectures and not to push the state-of-the-art of image classification accuracy. Our experimental results prove that GAT can be used to fuse feature vectors resulting from CNN architectures.