由RGB-D影像資訊產生多視角立體電影用之影像

Abstract

目前3D顯示技術已是很成熟的技術，然而目前可用的3D內容來源有限，造成推廣難度較高，即便用戶有經濟能力購買高檔3D影音設備也受限于可用的數位內容不夠多，讓3D功能等同虛設。本研究提出的將RGB+D 影像轉換成3D立體內容影像即為了舒緩部分立體內容不足的問題。 近年來有許多關於如何產生3D立體影像的研究，而從事這類研究，無可避免常會面臨到深度資訊估測以及產生新視角影像後修補的問題。深度估測的方式很多，有人工判斷、有根據深度線索判斷或是使用深度攝影機取得，人工判斷及根據深度線索判斷都相較耗時，尤其根據深度線索做判斷誤差的機率也較高。而深度攝影機則避免了深度資訊取得的困難，其誤差也較根據深度線索判斷來的低。然而使用深度攝影機取得的影像會有一個很大的問題，依照拍攝的情景不同，其影像可能會有不同程度的深度影像破洞，而深度資訊的完整及精準度直接影響了立體內容的好壞，因此需要對深度影像做合理的填補。而影像填補在過去亦有許多研究，主要考慮了顏色、紋理結構等因素，本研究使用兩種方式作處理，第一種是利用矩陣完整化技術的修補方式;第二種是使用以影像分割為基礎的修補方式，其中矩陣完整化是依據影像本身低rank的性質對影像做合理的填補，而基於影像分割的修補方式則是考慮顏色跟空間上位置的關係對影像作分割後再做填補。 實驗結果顯示我們的立體內容深度層次感相較於2D轉3D的立體內容明顯更佳，且相較于2D轉3D的技術，我們的研究使用的是深度攝影機所取得的深度資訊，因此深度資訊具有較低的誤差，產生出的立體影像能給予觀看者更好的立體感受。Nowadays, 3D display technology has been well developed and gradually became a matured technology. However, limited 3D contain resources obstruct this technology to be popularized to the market. Even if the customers can afford expensive media equipment, there is still lack of useable resources to function 3D display technology. This research provides the solution of converting RGB+D image to 3D image to partially improve the shortage of 3D resources. In recent decades, many researches are already working on how to create 3D images, which always involved depth measurement and generating image with another perspective. Depth measurement can be done by implementing the solutions such as manual judgments, depths cues, or using depth cameras. The former two solutions are relatively time consuming than the latter one. Especially the depths cues usually cause inaccuracy. Moreover, using depth cameras simplifies the difficulties of getting the depth data and decreases the inaccuracy as well. But there is a problem when using the cameras to collect the depth data, the images may have holes occurs which depends on shooting scenarios. The depth data need to be repaired under a reasonable condition because these two factors impact the 3D images’ qualities. In the past, solution to image inpainting has been proposed from many researches. The main considerations are about the colors and the texture. This research implements two methods to process the missing value of depth images. One is based on images’ low rank feature to use matrix completion technique; the other is based on image segmentation technique to do the depth image repairing. The results of experiment show that our 3D depth quality is obviously higher than the traditional 2D convert to 3D method. Furthermore, depth camera collects the depth data with higher accuracy so we can provide viewers a better experience in 3D display technology.