基於長短期記憶網路的疲勞檢測

Abstract

本論文重點介紹即時疲勞檢測流程。該系統將在 Python 內部完成這一切，並逐步構建它，以便能夠檢測到不同的姿勢，特別是困倦的跡象。 為了做到這一點，我們使用一些關鍵模型並使用 MediaPipe Holistic 來提取關鍵點。 這將使我們能夠從臉部提取關鍵點。 該系統使用 Tensorflow 和 Keras，並建立了一個長短期記憶模型 long short-term memory（LSTM），能夠預測螢幕上顯示的動作。我們需要做的是收集關於我們所有不同關鍵點的一些數據，所以我們收集我們臉上的數據並將它們保存為 Numpy 數據，以便處理多維的陣列或矩陣。人臉檢測方法基於一個深度神經網絡，使用 Sklearn 進行評估和測試，並使用 Matplotlib 幫助進行圖像可視化。能夠從臉部檢測到 468個地標，提取臉部的重要特徵並對數據進行變換，以便將數據導入 LSTM 模型。使用 LSTM 層繼續並預測時間分量，它能夠從多個幀預測動作，而不僅僅是單個幀。使用 Opencv 進行集成，然後使用網路攝影機進行即時預測。本研究成功使用 MediaPipe 與 LSTM 模型相結合，提出一套疲勞檢測的系統。實驗結果顯示，經機器學習後其檢測平均準確率能達到 90%。This paper focuses on the instant fatigue detection process. The system will do all this inside python and build it incrementally to be able to detect different poses, especially signs of drowsiness. To do this, we use some key models and use MediaPipe Holistic to extract keypoints. This will allow us to extract keypoints from the face. The system uses Tensorflow and Keras and builds a long short-term memory (LSTM) model that is able to predict actions displayed on the screen. What we need to do is collect some data about all our different keypoints, so we collect data on our faces and save them as Numpy data in order to work with multidimensional arrays or matrices. The face detection method is based on a deep neural network, evaluated and tested using Sklearn and aided in image visualization using Matplotlib. Able to detect 468 landmarks from faces, extract important features of faces and transform the data so that it can be imported into an LSTM model. Continuing and predicting the temporal component using an LSTM layer, it is able to predict action from multiple frames, not just a single frame. Integrate with Opencv, then use a webcam for instant prediction. This study successfully uses MediaPipe combined with LSTM model to propose a fatigue detection system. The experimental results show that the detection accuracy can reach an average of 90% after machine learning.