Effect of Generative Artificial Intelligence on Students' Learning Outcomes: A Quantitative Investigation in Higher Education
DOI:
https://doi.org/10.70670/sra.v4i2.2367Keywords:
Generative Artificial Intelligence, Chatgpt, Learning Outcomes, Higher Education, Academic Performance, Critical Thinking, Educational Technology, Quantitative Research, PunjabAbstract
Generative artificial intelligence (Gen-AI) has come into existence and is rapidly transforming the pedagogical world of Higher Education, with large language models (LLMs) like ChatGPT, Google Gemini and Microsoft Copilot at the forefront. Although there has been increasing uptake across under- and post-graduate students, there are current mixed and limited empirical findings on the exact nature, direction and magnitude of the impact of Gen-AI on measurable learning outcomes. This study fills this gap by exploring the correlation between the pattern of using Gen-AI and academic learning outcomes of students in HEIs in Punjab Province, Pakistan, by employing a quantitative method. A cross-sectional survey design was used to collect data from 521 undergraduate and graduate students from 5 major public universities with a validated 52 item self-report questionnaire that applied a latent factor analysis to the five constructs assessed: Gen-AI Utilization Frequency, Perceived Learning Enhancement, Critical Thinking Development, Academic Engagement, and Learning Outcome Achievement. Data were analyzed using descriptive statistics, Pearson's correlation analysis and multiple linear regression using SPSS-26. The result indicated that there is a statistically significant positive relationship between Gen-AI utilization and students' perceived learning outcomes (r = .71, p < .001), where Gen-AI Utilization Frequency (β = .38, p < .001) and Academic Engagement (β = .29, p < .001) were the most significant variables. The data also revealed that passive, unmonitored Gen-AI use was strongly negatively correlated with critical thinking scores (r = −.34, p < .001), indicating that excessive use of generative products may negatively affect higher-order thinking. The findings have implications for curriculum design, educational policymaking, and institutional technology policy, highlighting the need for pedagogically-informed integration of Gen-AI. This study employs a quantitative research approach. The research method used in this study is quantitative.
References
Ali, M., Khan, S. A., & Rahman, F. (2024). Awareness and use of artificial intelligence tools among university students in Pakistan: A cross-sectional study. Pakistan Journal of Education, 41(2), 88-107.
Alshater, M. M. (2022). Exploring the role of artificial intelligence in enhancing academic performance: A case study of ChatGPT. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4312358
Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52-62.
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 61(2), 228-239.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.
Dergaa, I., Chamari, K., Zmijewski, P., & Ben Saad, H. (2023). From human writing to artificial intelligence generated text: Examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, 40(2), 615-622.
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109.
Gerlich, M. (2023). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 13(1), 6.
Hayes, A. F. (2018). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd ed.). Guilford Press.
Kasneci, E., et al. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274.
Lund, B. D., & Wang, T. (2023). Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News, 40(3), 26-29.
Mollick, E. R., & Mollick, L. (2023). Assigning AI: Seven approaches for students, with prompts. SSRN Electronic Journal.
Rasul, T., et al. (2023). The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching, 6(1), 41-56.
Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading. Trends in Cognitive Sciences, 20(9), 676-688.
Shoufan, A. (2023). Exploring students' perceptions of ChatGPT: Thematic analysis and follow-up survey. IEEE Access, 11, 38805-38818.
Sok, S., & Heng, K. (2023). ChatGPT for education and research: A review of benefits and risks. SSRN Electronic Journal.
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257-285.
Tlili, A., et al. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15.
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89-100.
Zhai, X., et al. (2024). A review of artificial intelligence (AI) in education from 2010 to 2020. Complexity, 2021, 8812542.
