An Exploratory Study of EFL Learners’ Use of ChatGPT for Language Learning Tasks: Experience and Perceptions

Citation

  • Xiao, Yangyu, and Yuying Zhi. 2023. An Exploratory Study of EFL Learners’ Use of ChatGPT for Language Learning Tasks: Experience and Perceptions. Languages 8: 212.
  • DOI: https://doi.org/10.3390/languages8030212
  • Academic Editors: Jieun Kiaer and Ju‑Seong Lee
  • Received: 30 June 2023
  • Revised: 4 September 2023
  • Accepted: 5 September 2023
  • Published: 13 September 2023
  • Copyright: © 2023 by the authors.
  • Licensee MDPI, Basel, Switzerland.
  • Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Abstract

  • ChatGPT, an intelligent chatbot by OpenAI, presents both opportunities and challenges in language education.
  • It generates text, answers questions, and provides translations quickly, making it influential in AI.
  • The study investigates students’ experiences and perceptions of ChatGPT in language learning through interviews with five students at a top-tier international university in China.
  • ChatGPT can be a valuable learning partner and aid in language-related tasks.
  • Participants critically evaluate ChatGPT's output and modify prompts to enhance learning.
  • This critical judgment can offset potential threats to academic integrity.
  • The study supports ChatGPT as an effective tool for immediate feedback and personalized learning.
  • Findings suggest personalized guidance, technology-embedded language support, and lifelong learning skill development with ChatGPT.
  • Keywords: ChatGPT; language learning; learners’ perception; EFL learners

1. Introduction

  • Technology-enhanced learning tools have garnered attention for supporting language learning since ChatGPT's release in November 2022.
  • Technology offers opportunities for language learners to interact with the target language.
  • Authentic and interactive resources facilitate language skill development in a more immersive and meaningful way (Hassan Taj et al. 2017; Loncar et al. 2021).
  • Technology enhances learners’ motivation and autonomy.
  • Learners take control over their learning process (Gikas and Grant 2013; Peterson 2017).
  • AI-powered chatbots use natural language processing and machine learning to engage learners in dialogue-based interactions, offering personalized and interactive language learning experiences (Guo et al. 2022).
  • Chatbots adapt to learners’ proficiency levels, provide real-time feedback, and foster learner autonomy and self-correction (Chen et al. 2021).

2. ChatGPT in Language Learning

  • ChatGPT, developed by OpenAI, is an AI‑powered chatbot that utilizes large‑scale language models to generate human‑like text (Bender et al. 2021).
  • Potential benefits for language learning:
    • Assistance in language skill development.
    • Provision of personalized practice materials.
    • Support for writing, research, and problem‑solving tasks (Kasneci et al. 2023).
  • Concerns regarding potential risks:
    • Fairness.
    • Copyright infringement.
    • Breaches of academic integrity (Kasneci et al. 2023).
  • Criticisms have been raised regarding the accuracy and reliability of the information generated by ChatGPT.
  • There exists a notable research gap regarding the impact and effectiveness of ChatGPT from the learners’ perspective.
  • There is a need to explore how learners perceive and engage with ChatGPT, understanding their unique needs and challenges (Jeon 2021).
  • The current study addresses this research gap by examining learners’ perceptions of the role of ChatGPT when they complete language‑learning tasks.

2.1. The Role of ChatGPT in Technology-Enhanced Language Learning

  • Technology offers unique opportunities for language learners to engage with the target language (Hassan Taj et al. 2017).
  • One key advantage lies in the access to authentic and interactive resources provided by technology (Loncar et al. 2021).
  • Learners benefit from authentic language materials, such as videos, online articles, and podcasts, which reflect real‑life language use.
  • This exposure to authentic resources facilitates the development of learners’ language skills, enabling them to acquire proficiency in a more immersive and meaningful manner.
  • The use of technology has a positive impact on learners’ motivation in language learning (Gikas and Grant 2013; Smith 2018).
  • Technology empowers learners with a sense of autonomy and agency, allowing them to take control of their learning process (Peterson 2017).
  • Intelligent tutoring systems and adaptive language learning platforms have emerged (Heift and Chapelle 2012).
  • ChatGPT demonstrates the ability in generating text, answering questions, and completing various language‑related tasks (Kasneci et al. 2023).
  • ChatGPT can assist learners in developing language skills, such as writing and vocabulary acquisition, as well as providing personalized practice materials and explanations (Kasneci et al. 2023).
  • ChatGPT can aid in tasks related to writing, research reports, and problem‑solving while also offering discipline‑specific language skills.
  • Drawbacks related to ChatGPT:
    • Potential for cheating and its impact on assessment (Kohnke et al. 2023).
    • Responses are paraphrases of its sources without proper citations, which can lead to issues such as plagiarism.
    • The text created by ChatGPT can be inconsistent.
    • Responses generated by ChatGPT can vary significantly when identical prompts are input multiple times, indicating the accuracy and reliability of its outputs depend heavily on how prompts are worded (Kohnke et al. 2023).
    • Prompts are the initial text input to ChatGPT to provide context, while generated responses are the texts ChatGPT then produces.
  • Additional research into learners’ perspectives and understanding of these tools would further enrich our knowledge in this emerging field.

2.2. The Potential of AI-Powered Chatbots

  • AI‑powered chatbots have been recognized for their potential to deliver personalized and engaging language learning experiences.
  • Chatbots function as dialogue agents, engage in interactive conversations with human users, offering turn‑by‑turn interactions (Guo et al. 2022).
  • These virtual conversation partners facilitate interactive language exchanges and provide learners with real‑time feedback.
  • Chatbots can adapt to learners’ proficiency levels and personalize the learning content.
  • Shin et al. (2021) developed an adaptive chatbot that adjusts the difficulty of conversations based on learners’ language proficiency, resulting in improved engagement and learning outcomes.
  • Yang et al. (2022) found that learners who interacted with a pronunciation‑focused chatbot demonstrated significant improvement in pronunciation accuracy compared to a control group.
  • Chen et al. (2022) used a chatbot to support learners’ writing skills, offering automated feedback on grammar, vocabulary, and sentence structure, leading to improved writing performance over time.
  • Kim (2018) developed a vocabulary‑focused chatbot that presented learners with targeted vocabulary items in context and provided immediate feedback on their usage, resulting in enhanced vocabulary retention.
  • ChatGPT differs from traditional question‑and‑answer chatbots by engaging in back‑and‑forth conversations with users.
  • ChatGPT can summarize long articles or produce first drafts of presentations, providing new ideas for researchers and learners, and ChatGPT 4.0 even generates reasonably good quality academic articles on specific topics.
  • ChatGPT is trained on existing resources, and its responses are based on a selection of those resources.
  • Users may need to adjust their prompts to obtain the desired responses.
  • ChatGPT has limitations, including the potential to generate plausible but incorrect or made‑up responses.
  • ChatGPT needs to be trained with high‑quality data and the ability to understand complex prompts (Pavlik 2023).
  • Language resources provided by ChatGPT are restricted to pre‑2021 data, requiring users to critically evaluate responses derived from the chatbot.

3. Research Question

  • R.Q.: How do learners perceive the role of ChatGPT in English language learning?
  • Investigating learners’ perspectives is critical for:
    • Providing key information on how ChatGPT addresses or fails to address learners’ needs, preferences, and challenges.
    • Learner perceptions influence acceptance and integration of the technology.
    • Learner perspectives shape the ongoing ethical debates regarding AI.
  • Understanding learners’ perspectives is essential to optimizing the use of ChatGPT and addressing their specific needs and challenges in English language learning.

4. Research Methods

  • The current study employed a small‑scale exploratory approach to investigate students’ perceptions and experiences regarding the use of ChatGPT in the Chinese educational context.

4.1. Research Context

  • This study was conducted at a top‑tier English‑medium international university in China.
  • The university states specifically that by default ChatGPT should be prohibited in the completion of assignments that count towards the final grading of the course.
  • The study focused on students’ experiences of and perceptions towards ChatGPT 3.5, which was the free version of ChatGPT which our participants used.

4.2. Research Participants

  • The participants were five undergraduate students from diverse majors at a Chinese university.
  • Disciplines included Marketing, Translation, and Data Science across years 2–4.
  • Participants were purposefully chosen based on the first author’s classroom observations and regular interactions with students, ensuring all participants were active ChatGPT users.
  • Specific selection criteria included:
    • frequent ChatGPT usage strategies shared by Mike.
    • Tessa’s use of ChatGPT for IELTS preparation.
    • Dylan’s observed ChatGPT use for classwork.
    • Derek’s prior project on ChatGPT software.
    • Tere’s own research on ChatGPT and feedback.

4.3. Data Collection and Analysis

  • Data for this study were collected through semi‑structured interviews focusing on students’ views and experiences regarding the use of ChatGPT for language learning.
  • The interviews primarily addressed three key issues:
    • Students’ knowledge and understanding of ChatGPT.
    • How students utilized ChatGPT in language learning.
    • Students’ awareness of handling the drawbacks and challenges associated with the use of ChatGPT.
  • The interviews were conducted in Chinese, the participants’ mother tongue, and were transcribed by trained research assistants.
  • Data analysis were conducted thematically using NVivo 11, guided by the thematic map of the power of ChatGPT developed by Yan (2023).
  • This thematic map encompassed three key aspects:
    • The power of ChatGPT.
    • Potential challenges.
    • The proper use of ChatGPT.
  • Additionally, a fourth theme, “critical reflections on ChatGPT”, emerged from the findings.
  • The data were initially coded by the first author and subsequently reviewed and confirmed by the second author.

5. Findings

5.1. ChatGPT as a Peer Tutor for Providing Individualized Assistance

  • Three out of five students described ChatGPT as a peer tutor that provided support which was more easily accessible than that of teachers.
  • ChatGPT played the role of a language tutor by engaging in communication with students to facilitate improvement.
  • It seems that the ability to offer instant and easily accessible feedback is a key attractive feature of ChatGPT.
  • ChatGPT provides instructions on writing an effective “Introduction,” “Problems,” and “Solutions” section in a technical proposal.
  • The ChatGPT tutor has the ability to offer personalized feedback on whatever confused him.
  • ChatGPT offers personalized and adaptive learning experiences (Heift and Chapelle 2012; Guo et al. 2022).
  • Interactions with ChatGPT had helped them improve their grammar, vocabulary, and essay coherence.
  • ChatGPT contributed less or even restricted the development of language competence if students merely used it as a ghostwriter to complete the assignments for them.
  • ChatGPT primarily assisted with lower‑order language skills, such as basic vocabulary and grammar, while providing less support for advanced language skills, such as writing logical and authentic essays.
  • Language skills are better developed through extensive reading and listening rather than solely relying on ChatGPT interactions.
  • ChatGPT can improve the appearance of language but falls short of developing true language competence.

5.2. ChatGPT as a Source for Generating New Ideas

  • All five participants unanimously agreed that they would utilize ChatGPT to generate new ideas when planning or writing an English essay.
  • ChatGPT contributed to the planning stage when students completed a language writing task.
  • Although the ideas generated by ChatGPT were not necessarily innovative, they reminded her of ideas she had not come up with on her own.
  • The ideas generated by ChatGPT were somewhat broad and vague; therefore, she used them as a starting point and then looked for examples herself.
  • ChatGPT helped a lot with generating ideas; however, he felt that this was not something new, as he used Google to search for new ideas before the birth of ChatGPT.
  • ChatGPT just made the whole process more efficient by integrating all sorts of information.

5.3. Revising Prompts to Maximize the Learning Effects

  • All participants reached a consensus that they needed to constantly modify the prompts to obtain useful and accurate information.
  • ChatGPT needed constant training to function effectively, and she was aware of the successful strategies for a quality essay.
  • Students viewed ChatGPT as a technology‑enhanced tool and were actively engaged in exploring strategies and techniques to maximize the ben‑ efits of ChatGPT in helping them produce a better essay in English.

5.4. Making Critical Judgments on Information Generated by ChatGPT

  • Students reflected critically on the information generated by ChatGPT instead of accepting it automatically.
  • ChatGPT does not necessarily generate accurate information.
  • ChatGPT seldom said no to any questions and always made things up; therefore, they needed to judge if the information generated was usable.
  • Students only used ChatGPT as a source of information, and they still held critical attitudes toward its actual effectiveness.
  • Even when ChatGPT generated useful or accurate information, students did not automatically accept all of it.
  • ChatGPT was only used as a learning tool rather than a substitute for learning.
  • Students’ abilities to critically reflect on the capacity of ChatGPT seemed to be related to three resources:
    • Their understanding of the nature of ChatGPT.
    • Their own interactions and experimentation with ChatGPT.
    • The guidance provided by their teachers.

5.5. ChatGPT and Plagiarism Issues

  • All participants seemed to be aware of such a potential threat related to plagiarism.
  • For students with low motivation to learn, it is unavoidable that they would take a shortcut by using ChatGPT as a ghostwriter.
  • It was the course instructors’ responsibility to rethink what skills they should teach in the new technology era and guide students to use such a tool in a legiti‑ mate way.
  • Students believed that proper guidance from teachers was equally important.

6. Discussion and Conclusions

  • Students identified three key benefits associated with ChatGPT usage:
    • ChatGPT functions as a learning partner or personal tutor by providing personalized, easily accessible, and adaptive feedback.
    • It assists in improving language proficiency when students apply critical thinking skills, such as modifying prompts, training the model, and verifying and selectively accepting its outputs.
    • It facilitates idea generation for brainstorm‑ ing purposes.
  • Students demonstrated the capacity to think critically about the information generated by ChatGPT and reported their ability to modify prompts, train ChatGPT, verify, and selectively accept the information provided.
  • A significant ar‑ gument arising from this study is the necessity for teachers and educators to embrace this new AI tool and offer appropriate guidance to students.
  • Findings con‑ tribute positively by supporting the notion that ChatGPT can serve as an effective tool for providing immediate feedback and personalized learning experiences (Chen et al. 2022; Kim 2018).
  • ChatGPT functions as a personal language tutor for students.
  • Proper utilization of ChatGPT has the potential to enhance language competence.
  • Students developed a critical stance towards the role of ChatGPT in English lan‑ guage learning, which has not yet been explicitly addressed in existing literature.
  • Students consistently cri‑ tiqued the limited potential of AI in various aspects of learning and emphasized the need to judge if the information is usable.
  • To address the potential threats posed by ChatGPT and promote its productive use, teachers should ensure students are aware of the limitations of language models and encourage them to take responsibility for their own learning (Pavlik 2023).
  • The study generated pedagogical implications for the use of ChatGPT in English language teaching:
    • Our findings support the possibility of integrating technology‑ enhanced tools, such as ChatGPT, into the language classroom.
    • Our findings highlight the impor‑ tance of providing pedagogical guidance to help students utilize ChatGPT in a legitimate, proper, and productive manner (Yan 2023), instead of banning the tool completely.