運用微表情預測工作績效：卷積神經網絡的應用

Abstract

企業想要挑選高績效人才, 根本之道是透過有信度與效度的甄選工具。傳統的甄選流程大多透過履歷表篩選、面談評測應徵者的職能是否符合該職缺的需求。研究發現，應徵者的過去行為(Past Behaviour)是預測未來行為(Future Behaviour)與績效(Job Performance)最有效的預測因子，但行為事例式面談需耗費人力且缺乏效率。在心理學的領域發現，一個人在特定情境中的微表情除了反應當事人的情緒狀態外，也能用來預測當事人的未來行為傾向，從下一秒到下一年都有可能。隨著電腦視覺(Computer Vision)以及深度學習(Deep Learning)技術的發展，心理學家開始與電腦科技領域的專家合作，透過視訊記錄辨識當事人的微表情並用來推測當事人的未來行為。其中尤以卷積神經網絡(Convolutional Neural Network, CNN)，是目前最廣泛被應用在微表情分析的深度學習技術。本研究採用實證研究法，研究對象為個案公司中101位企業內部具業務工作性質員工。結果顯示，可以運用電腦視覺處理技術蒐集業務工作性質員工在特定環境中（模擬求職面試）回答特定問題所表現出來的面部微表情運動軌跡，以卷積神經網絡建立微表情與工作績效模型，具有91 %的機率可以推測其在工作績效的考核結果，提供企業選才時另一項快速且有效的甄選決策輔助工具。To select high-performance workforce, the fundamental way is to use selection tools based on the principle of credibility and validity. The traditional selection process mostly uses resume screening and interviews to evaluate whether candidate meet the job requirements.Studies found that job applicants’ past behaviors can predict future related behaviors and job performance. However, behavioral event interviews are labor-intensive and inefficient. In the field of psychology, it was found that a person’s Micro-Expressions in a specific situation can not only reflect the person’s emotional state, but also be used to predict the person’s future behavior tendency-to next second or even to next year. With the development of Computer Vision and Deep Learning technology (e.g. Convolutional Neural Network, CNN), psychologists are collaborating with computer science experts to decode targets Micro-Expressions through video recordings and use them to predict their behavior as well as emotion states. This research was conducted in a field environment, , and solicited 101 salespersons as participants from a company. The results show that sales representatives’ micro-expressions can be retrieved from mock job interviews in asynchronous video platform,, and the motional features of micro-expressions are associated with the targets’ job performance appraisals based the CNN modeling. Supported by accuracy levels of 91%, the study proves that a job interviewee’s micro-expressions can infer his or her job performance appraisal evaluated by his or her supervisor, and the proposed methodology can be used as an alternative personnel assessment tool for employment screening with more efficient and valid compared with the other traditional selection methods.