Investigating the Impact of AI-Facilitated Socratic Dialogue on Critical Thinking Skills

Abstract

This mixed-method study aims to explore the impact of having Socratic dialogue with artificial intelligence (AI) on students' critical thinking skills’ performance. The data were gathered from 35 foundation-level students. The study utilised a Watson Glaser critical thinking pre-test and post-test to measure their critical thinking abilities and a questionnaire to gather their perception of AI usage. During the intervention, students were engaged in structured AI-facilitated Socratic questioning sessions weekly. Data were analysed using paired-sample t-tests and linear regression, while survey responses were examined for thematic trends. Even though the data collected shows that students have a positive perception of using AI, there is no statistically significant difference between the pre- and post-tests (p > .05). The finding shows that students’ critical thinking performance did not improve throughout the study. Furthermore, the results for the linear regression (R² = 0.038, p > .05) show that there is very little correlation between students’ perception and their critical thinking performance. The study reveals the gap between students’ positive perception of using AI and their critical thinking skill development.