Fuzzy Weighted Support Vector Regression Using the Dual Coordinate Descent Method

Abstract

支持向量機(Support Vector Machine/SVM)是一種監督學習方法，通常用於分類問題。此外，SVM也被用於迴歸問題中，稱為支持向量迴歸（Support Vector Regression/SVR）。通過調整懲罰參數和可容忍的誤差界限，SVR比多數線性迴歸模型更有彈性。但其較常被用於單一結構的迴歸問題。 在無監督學習領域，聚類和混合回歸問題是我們非常感興趣的問題。因此，SVR也被推廣到混合回歸問題。基於模糊理論，延伸出了模糊加權支持向量迴歸(Fuzzy Weighted Support Vector Regression/FWSVR)。通過將隸屬度引入懲罰項，FWSVR可以處理混合迴歸問題，而不是像以前那樣一對一地使用SVR。然而，FWSVR在處理大規模數據時，所需的耗時較久。 在本文中，我們介紹了支持向量迴歸如何解決迴歸問題。並且我們將隸屬度作為不同數據的模糊權重，以構建模糊加權支持向量回歸（FWSVR）模型。然後，我們使用對偶坐標下降法找到模糊加權支持向量迴歸中拉格朗日乘數的更新函數。最後，我們考慮使用alpha cut方法來使模糊加權支持向量迴歸的結果更加有效率且穩定。實驗表明，FWSVR-DCD在處理大規模數據具有良好的性能且減少了所需的計算時間，並且估計結果對於有離群值的數據具有穩定性。Support Vector Machine is a kind of supervise learning method, which is commonly used for classification problems. In addition, SVM is also used in regression problems called Support Vector Regression (SVR). SVR is a more flexible method than some linear regression models, through tuning the tolerance parameters and the acceptance error margin. However, SVR is most used for the regression problem of a single structure. In the field of unsupervised learning, clustering and mixture regression problem are issues we are very interested in. Therefore, SVR is generalized to the mixture regression problems. Based on the fuzzy theory, the Fuzzy Weighted Support Vector Regression (FWSVR) is extended. By introducing the membership to the penalty term, FWSVR can handle the mixture regression problem at the same time, instead of using SVR one by one as before. However, FWSVR takes a long time when it processes large-scale data. In this thesis, we introduce how SVR solve the regression problems. And we consider the membership as the fuzzy weight of different data to construct the Fuzzy Weighted Support Vector Regression models. Then we use the dual coordinate descent method to find the updating equations of the parameters of FWSVR, this method called FWSVR-DCD. Finally, we consider the alpha cut method to make FWSVR more robust. The simulations show the FWSVR has good performance in large-scale cases and reduce the computing time, and the estimated results are robust for data with outliers.