基於非監督式生成對抗網路及對比學習之水下影像品質回復

Abstract

近年來水下環境之相關應用的重要性與日俱增，比如：水下資源探勘及水下環境監控。這些應用往往需要由水下無人載具來擷取水下數位影像資料以供後續之資料分析及其相關應用 (例如：水下物件偵測及水下影像分類等相關應用)。然而水下影像品質受到許多環境因素影響而造成影像退化，包括光線折射、反射等等，如此可能使得基於水下影像之相關應用無法得到良好的效果。近年來，隨著深度學習技術蓬勃發展，研究者提出許多基於深度學習的模型來改善水下影像的品質。目前現有方法中，以具備成對影像資料之監督式深度學習模型為主。成對影像學習雖然能以較輕量模型得到好的影像品質回復效果，但礙於現實難以取得成對的原始水下影像及其還原之陸上影像，因此模型訓練上受到許多限制。為了解決這個限制，許多研究以人工合成之影像來建立成對之訓練影像資料集。然而，人工合成之訓練影像資料集未必能反映真實的水下影像特性。為了解決此問題，最近已有研究提出使用生成對抗網路及非成對影像資料來進行深度學習網路訓練。本論文提出一基於非成對影像資料及生成對抗網路之深度學習模型，來處理水下影像回復的問題。本論文提出基於非成對訓練影像資料集及利用生成對抗網路架構訓練一影像領域轉換生成器將輸入之水下影像轉換為對應之陸上影像 (回復之水下影像)，其中我們利用對比學習及多樣損失函數來進行網路訓練。實驗結果已證實我們的方法可得到較佳的回復影像品質且優於 (或近似) 現有基於成對/非成對訓練資料之基於深度學習之水下影像回復網路。Nowadays, underwater applications play an important role in many fields, which increase the requirements of high-quality underwater images. However, underwater images may usually suffer from degeneration due to attenuation, color distortion, and noise from artificial lighting sources of imaging devices. To cope with the problem, single underwater image restoration has been popular recently. On the other hand, with the recently rapid development of deep learning techniques, several deep learning-based single underwater image restoration frameworks have been presented. Most of them rely on end-to-end supervised deep networks trained by synthesized paired image datasets, which may not fit real underwater image characteristics.To solve the problem that real paired underwater training image data are hard to collect, without needing paired training images, a novel unsupervised GAN (generative adversarial network)-based deep learning framework for single underwater image restoration is proposed in this thesis. To train the presented deep model, we introduce contrastive learning with perceptual, style loss, and other types of loss functions in our GAN-based structure to learn an image generator for translating underwater images to the corresponding in-air images. Extensive experiments have shown that the proposed method outperforms (or is comparable with) the state-of-the-art deep learning-based methods relying on paired/unpaired training data quantitatively and qualitatively.