探討AI生成與人工事實查核報告之語言風格差異——以新冠肺炎假新聞為例

Abstract

本研究旨在探討生成式 AI（Generative AI）生成的事實查核報告與人工撰寫的事實查核報告在語言風格上的差異，並以新冠肺炎假新聞為例進行比較。研究透過提示工程技術，設計了原型指令、思維鏈（Chain of Thought）和線索引導（Clue And Reasoning Prompting）三種不同的提示模板，使用 ChatGPT-4o 生成事實查核報告，並與臺灣事實查核中心的人工查核報告進行對比分析。研究從詞彙豐富度、句法複雜度、邏輯流暢性、關鍵字分佈和情感極性五個面向，利用自然語言處理工具進行分析。結果顯示，AI生成的查核報告在語言流暢性及一致性方面表現較佳，但在事實準確度及深度分析上仍有待改進，人工撰寫的報告則在專業性和語言靈活度方面表現更具優勢。本研究期望透過這些比較和分析，為生成式 AI 在事實查核領域的應用提供實證基礎，並提出改進生成模型語言風格的可能途徑，以期提高其在實際應用中的準確性和可靠性。This study aims to investigate the linguistic style differences between AI-generated and manually written fact-checking reports, with a focus on COVID-19 misinformation as a case study. This research utilized prompt engineering techniques to design three distinct prompt templates, including Vanilla, Chain of Thought (CoT), and Clue and Reasoning Prompting (CARP), and then it employed ChatGPT-4 to generate fact-checking reports. Subsequently, these AI-generated reports were compared to manually written reports from the Taiwan FactCheck Center using natural language processing tools. The comparative analysis concentrated on five linguistic dimensions:lexical richness, syntactic complexity, logical coherence, keyword frequency distribution, and sentiment polarity. The results indicate that AI-generated reports exhibit better performance in linguistic fluency and consistency. However, they still require improvement in terms of factual accuracy and depth of analysis. Conversely, manually written reports demonstrate greater advantages in professionalism and linguistic flexibility. It is hoped that this study contributes to provide empirical evidence supporting the application of generative AI in fact-checking through comparative analysis. It seeks to pave the way for enhancing the linguistic style of generative models, ultimately improving their accuracy and reliability in practical applications.