用於高光譜和多光譜影像融合的知識蒸餾師生網路

Abstract

近年來隨著太空探索的技術進步，太空遙測與感知領域變得越來越熱門。因為高解析度的高光譜影像在光譜帶上擁有更多的訊息，這些訊息對於遙測領域應用有很大的幫助，然而直接獲取高解析度高光譜影像會對硬體造成巨大的負擔。因此替代的方式是取得相同條件下的高解析度多光譜影像與低解析度高光譜影像，藉由此兩種影像的融合來獲得高解析度的高光譜影像。在本論文中，先是使用成對的高光譜和多光譜影像資料訓練一個較複雜的網路生成高解析度的多光譜影像和低解析度的高光譜影像融合結果，使用具有卷積感受野重複運用的RFRM模塊提取光譜訊息，再與多光譜影像擁有的空間信息融合生成最終結果。接著為了降低網路的大小，引入知識蒸餾的教師–學生架構建構一個小型的學生模型，讓學生模型去學習教師模型的特徵和資料集的訊息，進而達到效能與教師差距不大、但在速度以及模型複雜度上都優於教師模型的多光譜高光譜融合模型。經實驗顯示我們的蒸餾效果在影像融合成效上有很好的結果，並且在運行速度上相較教師網路快了近1.5倍，參數量則減少為原本的0.54倍。The field of space telemetry and sensing has become more and more popular in recent years with the technological advancement in space exploration. Because high-resolution hyperspectral images have more information in the spectral band, It is very helpful for remote sensing applications. However, direct acquisition of high-resolution hyperspectral images will impose a huge burden on the hardware. Therefore, the alternative is to obtain high-resolution multispectral images and low-resolution hyperspectral images under the same conditions, and get high-resolution hyperspectral images by fusing these two images.In this paper, we first train a complex network with paired hyperspectral and multispectral image data to generate fusion results of high-resolution multispectral images and low-resolution hyperspectral images. To reduce the size of the network, we introduce a teacher-student framework of knowledge distillation to construct a small student model, and let the student model learn the features of the teacher model and the information of the dataset. Generate a fusion model with performance comparable to the teacher model, but superior in terms of speed and complexity of the model.