Collaborative learning
Do educational chatbots raise project-course grades?
Open access · cc by · source: Europe PMC
In a Malaysian team design course, students using an educational chatbot outperformed a traditional class on learning achievement, without clear gains on motivation or creative self-efficacy.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Sixty Malaysian second-year students in two intact classes of 30 were assigned by class to chatbot-supported project-based learning or a traditional control sharing content, duration and instructor. The chatbot class scored higher on learning performance (µ = 42.500 vs 39.933; d = 0.978). Need for cognition, motivational beliefs, creative self-efficacy and perception of learning did not differ. Class-level assignment is not individual randomisation.
Evidence for the claim as stated.
Pre/post windows are not interchangeable designs. Peer feedback and faculty training are single-group rises. The ITS paper randomises after exclusions and adjusts for prior scores. Chatbots contrast two intact classes. Mentoring randomises classrooms and finds a null average with a subgroup gain. COVID UDL is a retrospective perception contrast, not a scored artefact. Calling all of them 'pre-test/post-test evidence that teaching improved' hides those differences.
Evidence for the claim as stated.
What moves is not always engagement. Chatbot students gained on course performance (d ≈ 0.98) without significant affective/motivation differences. Forty Iranian EFL learners randomised to text-based versus multimodal CMC for fourteen sessions split engagement by dimension (text-based stronger on autonomous, behavioural and cognitive; multimodal on emotional and social) with similar satisfaction. A pre/post 'engagement improved' headline would mis-state both papers.
Evidence for the claim as stated.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
Pre/post windows are not interchangeable designs. Peer feedback and faculty training are single-group rises. The ITS paper randomises after exclusions and adjusts for prior scores. Chatbots contrast two intact classes. Mentoring randomises classrooms and finds a null average with a subgroup gain. COVID UDL is a retrospective perception contrast, not a scored artefact. Calling all of them 'pre-test/post-test evidence that teaching improved' hides those differences.
- Supports · Online peer feedback that improves argumentative essays
- Supports · Faculty training that shifts teaching innovation attitudes
- Supports · Can a dialogue ITS fix fraction multiplication?
- Supports · Can showing shared interests improve student-teacher mentoring?
- Supports · Did remote COVID teaching feel less engaging to students?
What moves is not always engagement. Chatbot students gained on course performance (d ≈ 0.98) without significant affective/motivation differences. Forty Iranian EFL learners randomised to text-based versus multimodal CMC for fourteen sessions split engagement by dimension (text-based stronger on autonomous, behavioural and cognitive; multimodal on emotional and social) with similar satisfaction. A pre/post 'engagement improved' headline would mis-state both papers.
Discoveries this paper informs or conflicts with
- A course chatbot raised exam scores and left motivation exactly where it was
This paper informs this development.
Study at a glance
- Design
- Other — Quasi-experimental intact-class comparison of chatbot-supported vs conventional project-based learning
- N
- N=60 · Two classes of 30 students each (chatbot vs conventional)
- Population
- Malaysian students in a team design course
- Outcome
- Achievement (and secondary motivation/creativity measures) with educational chatbot support
Structured fields used in claim comparison tables when every cited study has a complete layer.
Inspect source passages
Do educational chatbots raise project-course grades?
These passages occur verbatim in the stored paper text. They support the points listed below; they do not verify every statement in the explanation.
60 students in two classes of 30
The 60 students were grouped into two classes (30 students per class) as either traditional learning class (control group-CT) or chatbot learning class (treatment group-EC).
What was the main learning-performance result?
hence indicating difference in learning achievement where the EC group outperformed the control group
Key findings
The chatbot group scored significantly higher on course learning performance than the control group, with a large Cohen’s d. Need for cognition, motivational beliefs, creative self-efficacy, and perception of learning did not differ significantly between groups.
Methodology
Sixty second-year TESOL/multimedia education students in Malaysia were assigned by class to a chatbot-supported project-based learning condition or a traditional control class. Both groups shared content, duration, and instructor; researchers compared learning achievement and several affective/perception measures.
Limitations
Class-level assignment (not individual randomization) limits causal certainty, and null effects on motivation measures mean chatbots were not a general engagement booster here. Findings are from one design course in one institution.
Related papers in this topic
Same topic cluster — not a recommendation engine.