The Role of Chatbots, Google and Online Dictionaries in Meaning Comprehension and Retention
Abstract
Today, language learners can turn to generative AI applications in situations of linguistic deficit, rather than search the web or consult dictionaries. The aim of the study is to find out if meaning comprehension and retention are affected by whether a chatbot, Google or a monolingual learners’ dictionary is consulted. Five online tools are investigated: ChatGPT-3.5, Microsoft Copilot, the Google English dictionary, the Longman Dictionary of Contemporary English (LDOCE) and the Collins COBUILD Advanced Learner’s Dictionary (COBUILD). In an online experiment, 128 upper-intermediate learners of English explained the meaning of 25 English words based on reference to the five tools. Meaning retention was tested in two post-tests: immediate and delayed. The results indicate that the tool significantly affects meaning comprehension as well as immediate and delayed retention. ChatGPT-3.5 and COBUILD, followed by Copilot, prove the most beneficial for decoding meaning and remembering it immediately afterwards. Google's English dictionary and LDOCE are the least helpful for these purposes. However, AI-generated explanations do not stick in memory for long. The best delayed retention results were obtained after reference to dedicated reference works: COBUILD and LDOCE, the latter being on a par with the Google dictionary. An analysis of the explanatory content offered by the tools suggests possible reasons for the observed regularities. The study implies that AI has not outshone dictionaries yet. While today bots successfully explain meaning and foster its immediate retention, they are not on a par with dictionaries as long-term learning aids.
Keywords: online dictionaries; Google; generative artificial intelligence; meaning comprehension; retention
Full Text:
PDFReferences
Albers, F., Trypke, M., Stebner, F., Wirth, J., & Plass, J. (2023). Different types of redundancy and their effect on learning and cognitive load. The British Journal of Educational Psychology, 93 (Suppl.2), 339-352. https://doi.org/10.1111/bjep.12592
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.
Cobuild. (n.d.). COBUILD advanced English dictionary. https://www.collinsdictionary.com/
Cumming, G., Cropp, S., & Sussex, R. (1994). On-line lexical resources for language learners: Assessment of some approaches to word definition. System, 22(3), 369-377.
Davis, D. (2024). Meet the 7 most popular search engines in the world. Search Engine Journal. Retrieved August 2, 2024, from https://www.searchenginejournal.com/seo/meet-search-engines/
de Schryver, G. (2023). Generative AI and lexicography: The current state of the art using ChatGPT. International Journal of Lexicography, 36(4), 355-387. https://doi.org/10.1093/ijl/ecad021
de Schryver, G., & Joffe, D. (2023, February 27). The end of lexicography, welcome to the machine: On how ChatGPT can already take over all of the dictionary maker’s tasks. [Paper presentation]. 20th CODH Seminar. Center for Open Data in the Humanities, Research Organization
of Information and Systems, National Institute of Informatics, Tokyo, Japan. https://youtu.be/watch?v=mEorw0yefAs
Diaz, M. (2023). How to use Copilot (formerly called BingChat). Retrieved August 2, 2024, from https://www.zdnet.com/article/how-to-use-the-new-bing-and-how-its-different-from-chatgpt/
Dziemianko, A. (2014). On the presentation and placement of collocations in monolingual English learners’ dictionaries: Insights into encoding and retention. International Journal of Lexicography, 27(3), 259-279. https://doi.org/10.1093/ijl/ecu012
Dziemianko, A., & Lew, R. (2013). When-definitions revisited. International Journal of Lexicography, 26(2), 154-175.
Hanks, P. (1987). Definitions and explanations. In J. Sinclair (Ed.), Looking up: An account of the COBUILD project in lexical computing (pp. 116-136). Collins ELT.
Jakubíček, M., & Rundell, M. (2023). The end of lexicography? Can ChatGPT outperform current tools for post-editing lexicography? In M. Medveď, M. Měchura, I. Kosem, J. Kallas, C. Tiberius, & M. Jakubíček (Eds.), Proceedings of the eLex 2023 conference (pp. 518-533). Lexical
Computing.
Kalyuga, S., & Sweller, J. (2021). The redundancy principle in multimedia learning. In R. Mayer & L. Fiorella (Eds.), The Cambridge handbook of multimedia learning (pp. 212-220). Cambridge University Press. https://doi.org/10.1017/9781108894333.021
Kamiński, M. (2021). Defining with simple vocabulary in English dictionaries. Amsterdam/Philadelphia: John Benjamins. https://doi.org/10.1075/tlrp.22
Kosem, I., Lew, R., Wolfer, S., Müller-Spitzer, C., & Silveira, M. (2019). The image of the monolingual dictionary across Europe: Results of the European survey of dictionary use and culture. International Journal of Lexicography, 32(1), 92-114. https://doi.org/10.1093/ijl/ecz002
Larsen-Freeman, D. (2012). On the roles of repetition in language teaching and learning. Applied Linguistics Review, 3(2), 195-210. https://doi.org/10.1515/applirev-2012-0009
Laufer, B. (1993). The effects of dictionary definitions and examples on the comprehension of new L2 words. Cahiers de Lexicologie, 63, 131-142.
Lew, R. (2023). ChatGPT as a COBUILD lexicographer. Humanities and Social Sciences Communications, 10(1), 704. https://doi.org/10.1057/s41599-023-02119-6
Lew, R., & Tomczak, E. (2022). Where should English dictionaries place multi-word expressions? A study of user consultation behaviour 3L: Language, Linguistics, Literature®, 28(3), 69-83. http://doi.org/10.17576/3L-2022-2803-05
Longman. (n.d.). Longman dictionary of contemporary English. https://www.ldoceonline.com/
Mackey, A., & Gass, S. (2016). Second language research: Methodology and design (3rd edition). New York: Routledge.
Mayer, R. (2009). Multimedia learning (2nd edition). Cambridge University Press.
Nichols, W. (2023, June 27-29). Invisible lexicographers, AI, and the future of the dictionary. [Paper presentation] eLex 2023 Conference: Electronic Lexicography in the 21st Century, Brno, The Czech Republic. https://www.youtube.com/watch?v=xYpwftj_QQI
Paas, F., & Sweller, J. (2012). An evolutionary upgrade of cognitive load theory: Using the human motor system and collaboration to support the learning of complex cognitive tasks. Educational Psychology Review, 24(1), 27-45. https://doi.org/10.1007/s10648-011-9179-2
Ptasznik, B. (2022). After the digital revolution: Dictionary preferences of English majors at a European university. Lexikos, 32, 220-249. https://lexikos.journals.ac.za/pub/article/view/1721
Ptasznik, B., Wolfer, S., & Lew, R. (2024). A learners’ dictionary versus ChatGPT in receptive and productive lexical tasks, International Journal of Lexicography, 37(3), 322-336. https://doi.org/10.1093/ijl/ecae011
Rees, G., & Lew, R. (2024). The effectiveness of OpenAI GPT-generated definitions versus definitions from an English learners’ dictionary in a lexically orientated reading task. International Journal of Lexicography, 37(1), 50-74. https://doi.org/10.1093/ijl/ecad030
Rundell, M. (2023). Automating the creation of dictionaries: Are we nearly there? In Y. An (Ed.), Proceedings of the 16th International Conference of the Asian Association for Lexicography (pp. 1-9). Yonsei University.
Statcounter. (2024). Search engine market share worldwide. Retrieved August 5, 2024, from https://gs.statcounter.com/search-engine-market-share
Sweller, J. (2005). Implications of cognitive load theory for multimedia learning. In R. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 19-30). Cambridge University Press. https://doi.org/10.1017/CBO9780511816819.003
Tal, S., Grossman, E., & Arnon, I. (2024). Infant-directed speech becomes less redundant as infants grow: Implications for language learning. Cognition, 249, 105817. https://doi.org/10.1016/j.cognition.2024.105817
Tal, S., Grossman, E., Rohde, H., & Arnon, I. (2023). Speakers use more redundant references with language learners: Evidence for communicatively-efficient referential choice. Journal of Memory and Language, 128, 1-11. https://doi.org/10.1016/j.jml.2022.104378
Tavast, A. (2023, June 27-29). Round Table on Large Language Models and AI in lexicography. [Panel session]. 8th Electronic Lexicography in the 21st Century Conference, Brno, Czech Republic. https://www.youtube.com/watch?v=dNkksTDYa_s
Refbacks
- There are currently no refbacks.
eISSN : 2550-2247
ISSN : 0128-5157