Collaborative learning
Do educational chatbots raise project-course grades?
Open access · cc by · source: Europe PMC
In a Malaysian team design course, students using an educational chatbot outperformed a traditional class on learning achievement, without clear gains on motivation or creative self-efficacy.
Study at a glance
- Design
- Other — Quasi-experimental intact-class comparison of chatbot-supported vs conventional project-based learning
- N
- N=60 · Two classes of 30 students each (chatbot vs conventional)
- Population
- Malaysian students in a team design course
- Outcome
- Achievement (and secondary motivation/creativity measures) with educational chatbot support
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
The chatbot group scored significantly higher on course learning performance than the control group, with a large Cohen’s d. Need for cognition, motivational beliefs, creative self-efficacy, and perception of learning did not differ significantly between groups.
Methodology
Sixty second-year TESOL/multimedia education students in Malaysia were assigned by class to a chatbot-supported project-based learning condition or a traditional control class. Both groups shared content, duration, and instructor; researchers compared learning achievement and several affective/perception measures.
Limitations
Class-level assignment (not individual randomization) limits causal certainty, and null effects on motivation measures mean chatbots were not a general engagement booster here. Findings are from one design course in one institution.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Sixty Malaysian second-year students in two intact classes of 30 were assigned by class to chatbot-supported project-based learning or a traditional control sharing content, duration and instructor. The chatbot class scored higher on learning performance (µ = 42.500 vs 39.933; d = 0.978). Need for cognition, motivational beliefs, creative self-efficacy and perception of learning did not differ. Class-level assignment is not individual randomisation.
Evidence for the claim as stated.
Pre/post windows are not interchangeable designs. Peer feedback and faculty training are single-group rises. The ITS paper randomises after exclusions and adjusts for prior scores. Chatbots contrast two intact classes. Mentoring randomises classrooms and finds a null average with a subgroup gain. COVID UDL is a retrospective perception contrast, not a scored artefact. Calling all of them 'pre-test/post-test evidence that teaching improved' hides those differences.
Evidence for the claim as stated.
What moves is not always engagement. Chatbot students gained on course performance (d ≈ 0.98) without significant affective/motivation differences. Forty Iranian EFL learners randomised to text-based versus multimodal CMC for fourteen sessions split engagement by dimension (text-based stronger on autonomous, behavioural and cognitive; multimodal on emotional and social) with similar satisfaction. A pre/post 'engagement improved' headline would mis-state both papers.
Evidence for the claim as stated.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
Pre/post windows are not interchangeable designs. Peer feedback and faculty training are single-group rises. The ITS paper randomises after exclusions and adjusts for prior scores. Chatbots contrast two intact classes. Mentoring randomises classrooms and finds a null average with a subgroup gain. COVID UDL is a retrospective perception contrast, not a scored artefact. Calling all of them 'pre-test/post-test evidence that teaching improved' hides those differences.
- Supports · Online peer feedback that improves argumentative essays
- Supports · Faculty training that shifts teaching innovation attitudes
- Supports · Can a dialogue ITS fix fraction multiplication?
- Supports · Can showing shared interests improve student-teacher mentoring?
- Supports · Did remote COVID teaching feel less engaging to students?
What moves is not always engagement. Chatbot students gained on course performance (d ≈ 0.98) without significant affective/motivation differences. Forty Iranian EFL learners randomised to text-based versus multimodal CMC for fourteen sessions split engagement by dimension (text-based stronger on autonomous, behavioural and cognitive; multimodal on emotional and social) with similar satisfaction. A pre/post 'engagement improved' headline would mis-state both papers.
Related papers in this topic
Same topic cluster — not a recommendation engine.