Research method
Quasi-Experimental Design
A quasi-experiment compares instructional conditions without the full control of a textbook RCT: historical cohorts, school-level assignment, intact classes, single-group pre/post, or randomisation after exclusions in a small local sample. The contrast can still be informative. It is not interchangeable with a multi-site randomised trial, and a significant post-test or engagement difference does not erase cohort, school or maturation confounds.
Engagement studies use quasi-experiments when they cannot (or do not) randomise individuals to WeChat versus forums, teacher-training schools, or faculty programmes. They answer 'did this non-equivalent contrast differ on the measured outcome?' Their main limitation is the leftover alternative explanation: semester, school, who volunteers, and n ≈ 9 all still sit in the estimate.
Evidence
What the evidence shows
Drawn from 6 studies in this library. Each finding starts with a plain-language takeaway, then the denser detail. Supports means evidence for a finding; Challenges means evidence against a stated position; Qualifies marks scope with a short note on each study’s contribution. Challenged positions are labeled — they are not findings.
In a Hong Kong flipped education course, a historical-cohort quasi-experiment compared WeChat mobile instant messaging with asynchronous online discussion. Twenty-six MIM and 28 AOD students completed the quiz; interviews supplemented behavioural, emotional and cognitive engagement indicators. MIM better facilitated interaction and, on optional tasks, more idea exchange and creating — stronger cognitive engagement when participation was voluntary. Cohorts were separated in time, so semester differences could confound the app contrast.
Sixty-six Taiwanese sixth-graders (35 experimental, 31 control, from 89 enrolled) were randomly assigned after exclusions to a dialogue ITS or traditional remedial teaching. The ITS group outperformed controls on the post-test after prior-score adjustment (F = 5.52); pretest α = 0.879. Random assignment here is still a short remedial session in one regional cohort, not a year-long curriculum RCT.
Universidad Francisco de Vitoria evaluated faculty training as a single-group quasi-experiment with 695 participants. Attitudes toward innovation and the LMS improved, and professors reported greater knowledge of innovative teaching elements, with broadly similar full-time and part-time patterns. No control group means maturation, self-selection and campus change remain live; student learning was not the outcome.
In two all-female high schools in Kuwait’s Al-Jahra district, one school’s teachers received Digital Education Shifting training (experimental school 746 students; control school 667). Experimental-school students recorded higher engagement in digital instructional environments. School-level assignment confounds training with everything else that differs between those two schools; achievement transfer was not established.
An eight-week graduate Distance and Online Education course compared minimally guided, task-oriented and idea-oriented scaffolds for triad collaborative problem-solving. Ten volunteers joined; one withdrew, leaving about nine analysed participants. Idea-centred scaffolding strengthened links among idea contribution, metacognitive regulation and knowledge-artefact behaviours in this sample. That n makes the ranking exploratory, not a definitive scaffold league table.
Sixty-two graduate students in 15 groups (four to five each) produced discussion logs over eight weeks; AI-supported prediction and SNA (density F = 4.90) plus 62 reflections were used to flag at-risk collaborators. Analytic comparisons of network metrics are not a multi-institution randomised trial of dashboards, and prediction is not a guarantee of later achievement.
Open questions
Tensions and limits
Some items are genuine disagreements on the same question. Others mark different assays, populations, or outcomes — limits on how far one study travels — not a forced fight between papers.
Unit of assignment and what 'worked' do not match. DES assigns schools (746 vs 667 students) and reports engagement. MIM assigns historical cohorts (26 vs 28 quiz completers) and reports interaction on optional tasks. The ITS paper randomises individuals after exclusions and reports an adjusted post-test. Faculty training has no comparison group. Scaffolding compares conditions among ~9 volunteers. These are not six estimates of one quasi-experimental engagement effect.
- Training teachers with DES boosts student engagement
- Mobile messaging can deepen course discussion engagement
- Can a dialogue ITS fix fraction multiplication?
- Faculty training that shifts teaching innovation attitudes
- Which scaffolding helps online collaborative problem-solving?
Study Role Design N Population Outcome Training teachers with DES boosts student engagement Supports OtherQuasi-experimental comparison of schools whose teachers received DES training vs control N=1413 · Experimental school 746 and control school 667 female high-school students Female high-school students in two Al-Jahra, Kuwait schools Student learning outcomes associated with Digital Education Shifting teacher training Mobile messaging can deepen course discussion engagement Supports OtherComparison of WeChat MIM vs asynchronous online discussion in a flipped course N=54 · 26 MIM and 28 AOD students completed the quiz Hong Kong flipped-course students using MIM or forum discussion Behavioral, cognitive, and emotional engagement under MIM vs AOD Can a dialogue ITS fix fraction multiplication? Supports OtherQuasi-experiment of dialogue-based fraction ITS vs traditional remedial instruction N=66 · 66 sixth graders analysed (35 ITS, 31 control) after exclusions from 89 enrolled Sixth-grade students in central Taiwan learning fraction operations Fraction multiplication/division achievement with ITS vs traditional remediation Faculty training that shifts teaching innovation attitudes Supports OtherPre–post faculty development program evaluating attitudes toward teaching innovation N=695 · 695 university professors participated University faculty (full-time and part-time) in a faculty training program Attitudes toward transformative/innovative teaching after the development experience Which scaffolding helps online collaborative problem-solving? Supports OtherSmall graduate-course comparison of CPS processes under three scaffolding types N=9 · 10 volunteers began; one withdrew mid-semester Chinese graduate students in a Distance and Online Education course Collaborative problem-solving process quality under idea-centered vs other scaffolds Outcomes disagree in kind. ITS moves a fraction post-test. DES and MIM target engagement/interaction. Faculty training moves self-reported attitudes. AI-analytics moves SNA metrics and offers a decision aid. Importing 'the quasi-experiment worked' across those endpoints overclaims.
- Can a dialogue ITS fix fraction multiplication?
- Training teachers with DES boosts student engagement
- Faculty training that shifts teaching innovation attitudes
- Can AI prediction dashboards help online groups?
Study Role Design N Population Outcome Can a dialogue ITS fix fraction multiplication? Supports OtherQuasi-experiment of dialogue-based fraction ITS vs traditional remedial instruction N=66 · 66 sixth graders analysed (35 ITS, 31 control) after exclusions from 89 enrolled Sixth-grade students in central Taiwan learning fraction operations Fraction multiplication/division achievement with ITS vs traditional remediation Training teachers with DES boosts student engagement Supports OtherQuasi-experimental comparison of schools whose teachers received DES training vs control N=1413 · Experimental school 746 and control school 667 female high-school students Female high-school students in two Al-Jahra, Kuwait schools Student learning outcomes associated with Digital Education Shifting teacher training Faculty training that shifts teaching innovation attitudes Supports OtherPre–post faculty development program evaluating attitudes toward teaching innovation N=695 · 695 university professors participated University faculty (full-time and part-time) in a faculty training program Attitudes toward transformative/innovative teaching after the development experience Can AI prediction dashboards help online groups? Supports OtherAI performance prediction and learning analytics applied to online graduate group work N=62 · 62 graduate students in 15 groups over an 8-week course Online graduate engineering students in collaborative writing groups Collaboration patterns and reflections under AI-supported performance prediction
Common misconceptions
A quasi-experiment is basically an RCT if there is a comparison group.
Historical MIM/AOD cohorts differ by semester. DES comparison is another school (667 vs 746), so peer composition, leadership and prior digital practice travel with the training. Those leftover differences are the point of calling the design quasi.
If sixth-graders were randomly assigned, the ITS result is a curriculum RCT.
Randomisation after exclusions in 66 students for a short remedial sitting still leaves durable transfer and control-quality unproven. It is a local contrast, not a multi-year curriculum trial.
Nine analysed volunteers can rank three scaffold types for all online collaborative problem-solving.
One withdrawal from ten volunteers in one eight-week course is an exploratory process comparison. The authors themselves treat idea-centred scaffolding as most useful in this sample, not as a universal ranking.
Exam-style questions
Short-answer questions that ask you to explain or compare, not recall.
Why does training teachers in one Kuwaiti girls’ school (746 students) versus leaving another school as control (667) block a clean claim that DES caused higher engagement?
The assigned unit is the school. Any school difference — staffing, prior digital practice, student body — is confounded with DES. Engagement, not achievement, was the outcome, and the sample is two all-female schools in one district.
MIM showed more idea exchange on optional tasks than AOD. What design feature should appear in a limitation sentence?
Cohorts were sequential (historical quasi-experiment), n completing the quiz was 26 versus 28, and the course used WeChat. Semester events, not only the app, could produce the optional-task difference; mandatory high-stakes discussion might not look the same.
Faculty attitudes improved among 695 trainees with no control group. Name two non-training explanations for the pre/post rise.
Maturation or repeated testing on the same attitude instruments; self-selection into the programme; concurrent campus LMS or innovation initiatives. Student learning was not measured, so even a real attitude shift would not demonstrate better teaching outcomes.
How should you distinguish the ITS paper’s random assignment from calling the AI-analytics course a quasi-experiment of dashboards?
The ITS paper assigns students to tutoring versus traditional remediation and compares adjusted post-tests. The analytics paper applies prediction and SNA to 62 students’ existing group work (15 groups) and compares network metrics; there is no randomised dashboard-off condition. Analytic comparison ≠ assigned teaching condition.
The studies
6 studies in this library bear on Quasi-Experimental Design, ordered by citations.
- Can AI prediction dashboards help online groups?
Combining AI performance prediction with learning analytics reshaped analysis of online graduate groups' collaboration in an engineering course.
- Mobile messaging can deepen course discussion engagement
Compared with forum-style discussion, WeChat-based mobile instant messaging better supported interaction, relationships, and higher-level cognitive engagement in optional tasks.
- Training teachers with DES boosts student engagement
Kuwaiti high-school students whose teachers were trained on a Digital Education Shifting model reported higher engagement than students of untrained teachers.
- Which scaffolding helps online collaborative problem-solving?
In a small Chinese graduate course, idea-centered scaffolding best strengthened connections among ideas, metacognitive regulation, and knowledge-artifact work during CPS.
- Can a dialogue ITS fix fraction multiplication?
Sixth graders using a dialogue-based math ITS outperformed peers who received traditional remedial instruction on fraction operations.
- Faculty training that shifts teaching innovation attitudes
A university faculty development program improved professors’ attitudes toward teaching innovation and LMS use, plus knowledge of innovative teaching elements.
Learn alongside