基於光學相干斷層掃描血管造影視網膜圖像的視覺預測多流網路

Abstract

視網膜前膜(Epiretinal Membrane，ERM)是一種慢性眼疾，肇因於視網膜的表面出現微細缺口，導致黃斑部增生一層纖維薄膜而影響視力。黃斑前膜手術為最典型治療方法，惟部分患者在手術後的視力恢復效果不佳，重要的因素之一是缺乏執行內限界膜(Inner Limiting Membrane，ILM)剝離時機的判斷，而此診斷障礙乃因為缺乏判斷黃斑前膜是否影響視力的標準，而導致醫生無法做出診斷，並於早期進行內限界膜剝離手術以提升術後的視力恢復。為了解決這個問題，本論文提出多種多流(multi-stream)神經網路，透過光學共輒斷層掃描(optical coherence tomography，OCT)、非侵入性光學共輒斷層血管掃描 (optical coherence tomography angiography，OCTA)、眼底螢光血管攝影(fundus fluorescence angiography，FFA)進行視力預測。我們收集454位患者上述三種影像並標記其視力資訊以訓練我們提出的多流神經網路，並以不同的影像輸入測試網路的效能。實驗結果顯示透過FFA全層、淺層、深層等三種影像在黃斑前膜患者的視力診斷中達到90.19%的準確性。最後，我們利用梯度權重類別活化映射(gradient-weighted class activation map，Grad-CAM) 可視化視力在OCT、 OCTA和FFA之間的特徵，幫助醫生進行診斷。Epiretinal Membrane (ERM) is a chronic eye disease. It is caused by tiny gaps on the surface of the retina, causing a fibrous membrane to proliferate in the macula, which affects vision. Macular epithelial membrane surgery is the most typical treatment method, but some patients have poor vision recovery after surgery. One of the important factors is the lack of judgment on the timing of performing Inner Limiting Membrane (ILM) peeling. This problem is due to the lack of criteria for judging how ERM will affect the visual acuity, which causes doctors cannot make the diagnosis to perform ILM peeling in early time to improve postoperative vision recovery.In order to solve this problem, this paper proposes a variety of multi-stream neural networks to find the correlation among visual acuity and optical coherence tomography (OCT), optical coherence tomography angiography (OCTA), fundus fluorescence angiography (FFA) images in ERMs patients. We collect the above three images of 454 patients and label their vision information to train our proposed multi-stream neural network, and then input different images to test the performance of the network. Experimental results show that by using three images of FFA full-thickness, superficial layer, and deep layer, the accuracy of the vision prediction of patients with epidermal membrane can reach 90.19%.Finally, we apply gradient-weighted class activation map (Grad-CAM) to visualize the characteristics of vision among OCT, OCTA, and FFA to help doctors make a diagnosis.