Chung-Ang University, Korea Military Academy team proves generative AI language tools' excellence through multi-dimensional learner evaluation

A collaborative research effort between Chung-Ang University and the Korea Military Academy has shed light on the effectiveness of generative AI in language learning. The study, led by Professor Lee Jang-ho of Chung-Ang University's English Education department and Professor Lee Han-sol of the Korea Military Academy, systematically compared the educational benefits of traditional corpus-based learning tools with those powered by generative AI.

Published in the latest issue of ReCALL, a leading academic journal in Computer-Assisted Language Learning (CALL), the research utilized a methodology called the Analytic Hierarchy Process (AHP). This approach allowed for a multi-dimensional evaluation of learning tools, moving beyond simple preference surveys to quantify the specific reasons and intensity of preference among learners.

The study involved university students who evaluated tools based on criteria such as sentence comprehension, relevance to meaning and syntax learning, perceived educational value, accessibility for independent use, and support for self-directed learning. The findings revealed a strong preference for generative AI tools, particularly those based on ChatGPT, which scored 2.6 times higher in overall preference compared to traditional corpus search tools.

“

The data-driven learning that had limitations in spreading outside the classroom due to its difficulty in use can now be provided in a much more accessible form to learners through generative AI, from the learner's perspective.

Learners cited "superior convenience" and "ease of autonomous learning" as the primary advantages of AI-based tools. Specifically, the AI tools received a score 3.76 times higher for accessibility, indicating their potential to overcome the complexity often associated with traditional language learning programs. This suggests that generative AI can make language learning more accessible and manageable for individuals in their daily lives.

Professor Lee Jang-ho noted that while the results highlight the learner-centric advantages of generative AI in making data-driven learning more accessible, further research is needed to measure actual learning outcomes. He cautioned against overstating the findings, acknowledging that learner curiosity about new technology might have influenced the results. The research team plans to conduct follow-up studies to validate these findings through actual learning performance metrics.

“

However, these results are based on learner perception, and there is a possibility that curiosity about new technology has been reflected, so follow-up experimental research measuring actual learning outcomes needs to be supplemented.