運用生成式AI進行程式設計之研究

Abstract

本研究旨在探討學生如何使用生成式AI—ChatGPT進行程式設計任務的歷程，並更進一步去探討運算思維使用傾向、程式設計自我效能、程式設計的先備知識對學生進行任務的行為模式有什麼樣的關聯。本研究採用混和研究的方式，並採用便利取樣，以就讀大專院校並修習過程式設計者為對象，招募男性與女性各20名為本研究的受試者。在資料收集過程方面，本研究先以問卷調查的方式，量測學生的運算思維使用傾向、程式設計自我效能、程式設計的先備知識，隨後再請學生利用ChatGPT進行程式設計任務，並全程錄影其任務過程。在資料分析方面，本研究先以內容分析的方式，分別歸納出受試者的任務行為和使用提示詞目的之主要類別，再針對這些類別和運算思維使用傾向、程式設計自我效能、程式設計的先備知識進行相關性的分析。研究結果發現，受試者的運算思維使用傾向、程式設計自我效能和程式設計的先備知識與其ChatGPT的使用行為呈現顯著負相關；但與手動撰寫程式碼的時間則呈現顯著正相關。此外，受試者使用提示詞的目的，程式碼再生成-不符預期結果之使用次數與其程式設計自我效能、程式設計的先備知識亦呈現顯著負相關。再者，本研究根據受試者的任務行為，透過階層分群法將其分為三類，即擅長AI溝通者、不擅長AI溝通者、手動程式設計者。最後，本研究透過Kruskal-Wallis統計法來檢核不同執行任務類型之受試者在相關變項上的差異；分析結果發現，不擅長AI溝通者的學生，程式設計任務總分是三群中最低的，並且其先備知識和程式設計自我效能之邏輯思考、演算法、除錯的分數，皆顯著低於手動程式設計者。This study aimed to explore how students used generative AI, specifically ChatGPT, to complete programming tasks and the relationship among students' computational thinking tendencies, programming self-efficacy, prior knowledge of programming, and their behaviors of using ChatGPT. This study adopted a mixed-methods design and recruited 40 college students to participate in this study. The participants first completed the Computational Thinking Tendency Survey (CTTS), the Computer Programming Self-Efficacy Survey (CPSES), and a pretest on programming knowledge. Next, they were asked to utilize ChatGPT to complete the two programming tasks, throughout which their manipulating behaviors were video recorded. Regarding data analysis, this study first used content analysis methods to categorize the participants' behaviors of using ChatGPT to complete the tasks and their objectives of generating prompts. Then, correlation analyses were conducted to examine the relationships among their behaviors of using ChatGPT, their objectives of generating prompts, and their CTTS, CPSES, and pretest scores. The findings reveal that the participants' CTTS, CPSES, and pretest scores were significantly negatively correlated with their behaviors of using ChatGPT but positively correlated with the duration of time theyspent manually writing codes. In addition, the frequency of writing prompts to regenerate codes was significantly negatively correlated with the participants’ CPSES and pretest scores. Moreover, this study categorized the participants into three groups based on the results of hierarchical clustering analysis of their behaviors of using ChatGPT: AI communicators (skilled in AI interaction), non-AI communicators (less skilled in AI interaction), and manual programmers. In sequence, the Kruskal-Wallis statistical tests were conducted to examine the differences among the three groups in related variables. The results indicate that the non-AI communicators had the lowest performance in programming tasks, and their CTTS and CPSES scores were significantly lower than the manual programmers.