基於 SwinTransformer 及深度學習網路之高光譜影像融合

Abstract

高光譜影像(Hyperspectral Image)以及多光譜影像(Multispectral Image)融合常被用來解決高光譜影像問題，旨在融合低解析度高光譜影像(LRHSI)以及高解析度多光譜影像(HRMSI)，是目前最常見的方法之一，通常高光譜影像的空間解析度較低，且直接取得高解析度之高光譜影像具有高昂的成本，而透過融合獲取高解析度高光譜影像是一種經濟實惠的方法。在影像處理領域融合方法是一種關鍵技術，因為高解析高光譜影像很好的促進了遠程材料辨識及分類任務，從而在衛星遙感領域獲得很大的關注。在衛星遙感領域很少有人嘗試使用Transformer，而Transformer在很多高級視覺任務中表現出驚人的成果，在本文中，我們提出了處理HSI/MSI融合任務的網路模型，基於SwinTansformer以及深度卷積網路(DCNN)之融合網路，稱為SwinDFN，SwinDFN由兩個部分組成：1)傳統卷積神經網路對HSI以及MSI影像初步融合，其中引入了Depthwise卷積技術來更有效地結合 HSI 和 MSI 之間的光譜響應函數以及對網路參數量做壓縮，2)具有殘差結構的SwinTansformer特徵提取模塊，來對影像特徵做增強，所提出之方法實現了基於規模較小的網路達到較好的HSI/MSI融合性能。Hyperspectral image and multispectral image fusion are often used to solve hyperspectral image problems, aiming at fusing low-resolution hyperspectral image (LRHSI) and high-resolution multispectral image (HRMSI), which is currently the most common one of the methods. Generally, the spatial resolution of hyperspectral images is low, and obtaining high-resolution hyperspectral images directly is costly, while obtaining high-resolution hyperspectral images through fusion is an economical and affordable method. The fusion method is a key technology in the field of image processing, because high-resolution hyperspectral images can well promote remote material identification and classification tasks, thus gaining great attention in the field of satellite remote sensing. Few people have tried to use Transformer in the field of satellite remote sensing, and Transformer has shown amazing results in many advanced vision tasks. In this paper, we propose a network model for processing HSI/MSI fusion tasks, based on the fusion network of SwinTanformer and deep convolutional network, called SwinDFN, and SwinDFN consists of two parts: 1) The traditional convolutional neural network initially fuses HSI and MSI images, which introduces Depthwise convolution technology to more effectively combine the spectral response function between HSI and MSI and compress the network parameters. 2) SwinTansformer feature extraction module with residual structure to enhance image.