Overview
For Task 1, we were assigned to make a comparative report charting our research methodologies and the differences in resource outputs between LLM chatbots such as ChatGPT and traditional digital search engines such as UBC’s Summon database.
Premise
The question that all the keywords were derived out of was “How can digital textbooks be effectively employed in K-12 classrooms to promote active learning for EFL learners? From there on, the keywords extracted were digital textbooks, EFL, and active learning. Additionally, metacognition, language learning and play based learning were added to the list of keywords in order to expand the scope of materials relevant to the efficacy of digital textbooks in an EFL setting.
By using the ERIC (EBSCO) Subjects functionality, adjacent keywords such as english as a foreign language, metacognition, gamification, and electronic textbooks were also extracted.
In contrast, ChatGPT allows natural language queries such as “How can digital textbooks be effectively employed in K-12 classrooms to promote active learning for EFL learners?” and further ask for clarification for its conclusion with the prompt “can you provide me sources on where you might have gotten the conclusion “Digital textbooks promote active learning for EFL learners when they go beyond static content to support interaction, collaboration, personalization, and reflection. The teacher’s role is crucial in designing activities that exploit these features for authentic language use.”
| Keywords used |
|---|
| digital textbooks |
| EFL, English as a foreign language, language learning |
| active learning, metacognition, play based learning, gamification |
Search Results
| Summon | ERIC (EBSCO) | ChatGPT |
|---|---|---|
| The Effects of Digital Textbooks on Students’ Academic Performance, Academic Interest, and Learning Skills. | Moving forward: A framework for technology and TBLT. | |
| Digital textbooks are useful but not everyone wants them: The role of technostress. | Impact of an EFL Digital Application on Learning, Satisfaction, and Persistence in Elementary School Children. | |
| Impact of an EFL Digital Application on Learning, Satisfaction, and Persistence in Elementary School Children | Enhancing EFL learners’ engagement in situational language skills through clustered digital materials. | |
| Untangling the Relationship between AI-Mediated Informal Digital Learning of English (AI-IDLE), Foreign Language Enjoyment and the Ideal L2 Self: Evidence from Chinese University EFL Students. | Teachers’ Collaborative Reflections on Classroom Materials: From Traditional Consumers to Digital Materials Developers and Teacherpreneurs. | |
| Encouraging Metacognition in Digital Learning Environments. | EFL teachers’ perceptions of the use of e-textbooks at secondary schools in Dong Thap province |
ERIC(EBSCO) was useful in being able to provide antonyms and alternative keywords but ultimately provided the least relevant information. While ERIC was able to retrieve a plethora of information regarding EFL, it could not find much when AND digital textbooks was added as a query.
Summon was the most comprehensive and relevant in terms of search results. Utilizing the filters on Summon helped narrow or extend search prompts greatly, offering practical and relevant journal articles and book sections with topics that intersected between ICT as well as educational practices.
ChatGPT was easiest to use, as natural language models required little in terms of whittling away at combinations of keywords. Conversely, it was also the most opaque in terms of clarity as ChatGPT does not cite ideas as an article or book might. Upon being prompted to reveal its sources for the conclusions, ChatGPT provided a list of five to six sources that were valid (not hallucinations), but could not correlate exactly which areas of the papers it was applying to which idea (once again, no in-text citations).
Validity of sources
The validity of each source has been assessed based on the following categories: source authority, peer review status, publication relevancy, transparency and citations, bias and objectivity.
Based on the results that were provided, if one or more articles from either Summon/ERIC or ChatGPT were questionable, the search result parameters were marked as “invalid”
| Chat GPT | Summon & ERIC EBSCO | |
|---|---|---|
| Source authority | valid | valid |
| Peer review status | invalid | valid |
| Publication relevancy | valid | valid |
| Citations | valid | valid |
| Bias and objectivity | invalid | invalid |
notes on source authority
No clear sources of disrepute were noted. All journals cited by ChatGPT as well as search results on Summons had clear associations with well established journals.
notes on peer review status for reflection:
ChatGPT does not verify if a journal paper or book section is peer reviewed. That is something you have to check the Summons database for. Conversely, Summon did feature peer reviewed papers which offered dubious research so peer review might not necessarily be an absolute standard for the quality of a research paper (see below, Reflections para 8)
notes on publication relevancy:
All publications were 10 years or under. Both Summon and ChatGPT offered extremely recent papers at the top of the search list, some as recent as a few months ago.
notes on citations:
All publications were diligent with their citations, offering extensive lists of their references on their DOI pages. Some DOI urls have been locked from public access hence they have been removed from the reference list.
notes on bias and objectivity:
From the papers provided by the databases or Chatgpt, the primary qualifier for a valid assessment was if the research methodologies and subsequent findings were extrapolated clearly and the findings had clear and obvious connections to the methodology. While bias might be difficult to be rid of in any setting, the objective clarity of research conducted would validate the paper.
Reflections
The research topic I chose to work with was regarding the efficacy of digital textbooks and possible strategies available to teachers in the EFL space. As my job currently requires me to be deeply involved with the use of digital textbooks in classrooms, this felt like a highly appropriate topic for me to investigate. Digital textbooks have been touted for their strengths and benefits and their adoption across Japan has been widespread. There have been a plethora of ways in which Japanese teachers of English have been employing digital textbooks but without a concrete strategy or methodology. This has led to some rather obfuscated practices in classroom without clear outcomes (eg: shadow reading as a class) in linguistic skill or knowledge growth. Anecdotally speaking, digital textbooks have been difficult for me to adopt in my classrooms due to the fact that the primary goals of my lessons require L2 output either in writing or spoken form.
After some discussions with fellow teachers I constructed the primary goal of the research which is How can digital textbooks be effectively employed in K-12 classrooms to promote active learning for EFL learners?
The thought process behind this was that digital textbooks in Japan (Here We Go, Big Dipper) primarily serve the teacher’s workflow and are less concerned with student-facing learning. Access to multimedia such as audio and video are useful for L2 teachers who don’t often have the resources they need, however its daily use for study and practice was largely focused on passive learning. Thus “active learning” was appended to the research question.
From there on, UBC’s Summons database was able to provide a plethora of resources from which I was able to ascertain how far the digital textbook field overlapped with other learning outcomes, thus also incorporating ‘metacognition’ and ‘gamification’ into my keyword searches.
Summons versus ChatGPT, the search experience:
LLMs dont require you to be nearly as precise with your keywords as it feels intuitive to have a conversation about what you need. I was able to narrow down my concerns and consider the implications of my search as I progressed along my search. ChatGPT offered me conclusions and implications.
✅ In short: Digital textbooks promote active learning for EFL learners when they go beyond static content to support interaction, collaboration, personalization, and reflection. The teacher’s role is crucial in designing activities that exploit these features for authentic language use. (OpenAI, 2025)
After which I asked ChatGPT to reveal its sources. It did so quite effectively and offered to pull direct quotes from the papers to strengthen its evidence for the benefits of using digital textbooks in classrooms. The caveat to this is that it is not offering citations, but rather pulling quotes to strengthen its case. The slim difference between providing citations to give clarity for statements in the body of the discussions, versus pulling quotes to double down on a persuasive argument creates a gap for misinterpretation, leaving the cited work susceptible to biased interpretation.
I was unable to locate a specific chapter or page that directly discusses “digital personalization and informal e-learning” in Task-Based English Language Teaching in the Digital Age: Perspectives from Secondary Education by Valentina Morgana. The sources I found (book descriptions, tables of contents, publisher listings) do not include detailed previews or content snippets that would allow verification of that exact phrase. (OpenAI, 2025)
For these reasons, I find ChatGPT to be an effective search assistant, however, its claims and conclusions to be relatively unverifiable. Most likely, I would like to employ it as a preliminary filter to be able to pull more specific search results from Summon or a similar database.
Having to scour through Summon however, was like working muscles I didn’t know I had. After trying multiple keyword combinations, I was finally able to create a list that I felt were relevant to the topic. Not being well versed with verifying credentials of scholarly publications, it was definitely a bigger hassle having to vet things such as peer review, source authority etc. from the get go. It slowed down my search quite a lot. However, since I had reasonable doubt towards the insights that ChatGPT had seemingly derived from its sources, I ended up having to vet ChatGPTs sources as well, so the time investment probably evened out on both sides.
Validity of sources
The first paper I read was retrieved by Summon and had all the markings of a valid article. Titled ‘The Effects of Digital Textbooks on Students’ Academic Performance, Academic Interest, and Learning Skills’ (Lee et al., 2023) I spent an hour poring over its methodology and conclusion and finally came to the conclusion that I learned nothing. It failed to specify the parameters of its research relying on oral explanations of what they do, but not actually explaining what they did, instead citing sources from over 30 years ago (Lee et al., 2023. pp 799, para7). The conclusion was a ridiculous tangential prediction about the effectiveness of digital textbooks on students learning by associating the effect with students potential lifetime earnings (Lee et al., 2023. pp 806, para1).
Thankfully, this was the first paper I read and immediately came to the conclusion that peer review could be flawed and despite having valid sources and coming from a valid authority, did not mean a paper’s bias and objectivity was also accounted for. Frankly speaking, it felt like state propaganda literature. (Lee et al., 2023. pp795, para 6. pp 796, para 3. pp798, para 7.)
Conclusion,
Perhaps due to my skepticism of ChatGPT, I was much more critical in the way I approached my searches and how thorough I was with its vague proclamations. Both ChatGPT and traditional database searches are immensely useful in allowing users to connect with material they would have never thought of looking through, but in the end, the appropriate application of both are dependent on how adept the reader is at identifying red flags.
ChatGPT
| Queries |
|---|
| How can digital textbooks be effectively employed in K-12 classrooms to promote active learning for EFL learners? |
| can you provide me sources on where you might have gotten the conclusion “Digital textbooks promote active learning for EFL learners when they go beyond static content to support interaction, collaboration, personalization, and reflection. The teacher’s role is crucial in designing activities that exploit these features for authentic language use.” |
| ChatGPT Citation References |
|---|
| Salih, Abdelrahman Abdalla; & Omar, Lamis Ismail. (2024). Enhancing EFL learners’ engagement in situational language skills through clustered digital materials. Frontiers in Education (Lausanne), 9. https://doi.org/10.3389/feduc.2024.1439104 |
| Dinh, T. H., & Le, X. M. (2020). EFL teachers’ perceptions of the use of e-textbooks at secondary schools in Dong Thap province. CTU Journal of Innovation and Sustainable Development, 12(2), 15-24. https://doi.org/10.22144/ctu.jen.2020.011 |
| Morgana, V. (2025). Moving forward: A framework for technology and TBLT. In Task-Based English Language Teaching in the Digital Age (1st ed., pp. 41–60). Bloomsbury Publishing. |
| Tajeddin, Z., & Asadnia, F. (2023). Teachers’ Collaborative Reflections on Classroom Materials: From Traditional Consumers to Digital Materials Developers and Teacherpreneurs. RELC Journal, 56(2), 287-304. https://doi.org/10.1177/00336882231222036 (Original work published 2025) |
| Hori, R., Fujii, M., Toguchi, T. et al. Impact of an EFL Digital Application on Learning, Satisfaction, and Persistence in Elementary School Children. Early Childhood Educ J 53, 1851–1862 (2025). https://doi.org/10.1007/s10643-024-01653-5 |
UBC Library (Summon & ERIC)
| Citation References |
|---|
| Lee, S., Lee, J.-H., & Jeong, Y. (2023). The Effects of Digital Textbooks on Students’ Academic Performance, Academic Interest, and Learning Skills. Journal of Marketing Research, 60(4), 792–811. https://doi.org/10.1177/00222437221130712 |
| Verkijika, S. F. (2019). Digital textbooks are useful but not everyone wants them: The role of technostress. Computers and Education, 140, 103591. |
| Liu, L., & Hwang, G. (2024). Effects of metalinguistic corrective feedback on novice EFL students’ digital game‐based grammar learning performances, perceptions and behavioural patterns. British Journal of Educational Technology, 55(2), 687–711. https://doi.org/10.1111/bjet.13400 |
| Hori, R., Fujii, M., Toguchi, T., Wong, S., & Endo, M. (2025). Impact of an EFL Digital Application on Learning, Satisfaction, and Persistence in Elementary School Children. Early Childhood Education Journal, 53(5), 1851–1862. https://doi.org/10.1007/s10643-024-01653-5 |
| Devers, C.J., Devers, E.E., Oke, L.D. (2018). Encouraging Metacognition in Digital Learning Environments. In: Ifenthaler, D. (eds) Digital Workplace Learning (pp. 9-22). Springer, Cham. https://doi.org/10.1007/978-3-319-46215-8_2 |
Additional Citations
OpenAI. (2025). ChatGPT (Sept 12 version) [Large language model]. https://chatgpt.com/share/68c3d0b1-3e30-8005-9cb1-32b872179ce5