Research method
Quasi-Experimental Design
6 studies
A quasi-experiment compares instructional conditions without the full control of a textbook RCT: historical cohorts, school-level assignment, intact classes, single-group pre/post, or randomisation after exclusions in a small local sample. The contrast can still be informative. It is not interchangeable with a multi-site randomised trial, and a significant post-test or engagement difference does not erase cohort, school or maturation confounds.
Engagement studies use quasi-experiments when they cannot (or do not) randomise individuals to WeChat versus forums, teacher-training schools, or faculty programmes. They answer 'did this non-equivalent contrast differ on the measured outcome?' Their main limitation is the leftover alternative explanation: semester, school, who volunteers, and n ≈ 9 all still sit in the estimate.
Studies
6
Findings
6
6 supporting · 0 challenging · 0 qualifying citations
Open tensions
2
Currently
What we know
- In a Hong Kong flipped education course, a historical-cohort quasi-experiment compared WeChat mobile instant messaging with asynchronous online discussion. Twenty-six MIM and 28 AOD students completed the quiz; interviews supplemented behavioural, emotional and cognitive engagement indicators. MIM better facilitated interaction and, on optional tasks, more idea exchange and creating — stronger cognitive engagement when participation was voluntary. Cohorts were separated in time, so semester differences could confound the app contrast.
- Sixty-six Taiwanese sixth-graders (35 experimental, 31 control, from 89 enrolled) were randomly assigned after exclusions to a dialogue ITS or traditional remedial teaching. The ITS group outperformed controls on the post-test after prior-score adjustment (F = 5.52); pretest α = 0.879. Random assignment here is still a short remedial session in one regional cohort, not a year-long curriculum RCT.
- Universidad Francisco de Vitoria evaluated faculty training as a single-group quasi-experiment with 695 participants. Attitudes toward innovation and the LMS improved, and professors reported greater knowledge of innovative teaching elements, with broadly similar full-time and part-time patterns. No control group means maturation, self-selection and campus change remain live; student learning was not the outcome.
- In two all-female high schools in Kuwait’s Al-Jahra district, one school’s teachers received Digital Education Shifting training (experimental school 746 students; control school 667). Experimental-school students recorded higher engagement in digital instructional environments. School-level assignment confounds training with everything else that differs between those two schools; achievement transfer was not established.
- An eight-week graduate Distance and Online Education course compared minimally guided, task-oriented and idea-oriented scaffolds for triad collaborative problem-solving. Ten volunteers joined; one withdrew, leaving about nine analysed participants. Idea-centred scaffolding strengthened links among idea contribution, metacognitive regulation and knowledge-artefact behaviours in this sample. That n makes the ranking exploratory, not a definitive scaffold league table.
Largest unresolved question
Unit of assignment and what 'worked' do not match. DES assigns schools (746 vs 667 students) and reports engagement. MIM assigns historical cohorts (26 vs 28 quiz completers) and reports interaction on optional tasks. The ITS paper randomises individuals after exclusions and reports an adjusted post-test. Faculty training has no comparison group. Scaffolding compares conditions among ~9 volunteers. These are not six estimates of one quasi-experimental engagement effect.
Common misconceptions
A quasi-experiment is basically an RCT if there is a comparison group.
Historical MIM/AOD cohorts differ by semester. DES comparison is another school (667 vs 746), so peer composition, leadership and prior digital practice travel with the training. Those leftover differences are the point of calling the design quasi.
If sixth-graders were randomly assigned, the ITS result is a curriculum RCT.
Randomisation after exclusions in 66 students for a short remedial sitting still leaves durable transfer and control-quality unproven. It is a local contrast, not a multi-year curriculum trial.
Nine analysed volunteers can rank three scaffold types for all online collaborative problem-solving.
One withdrawal from ten volunteers in one eight-week course is an exploratory process comparison. The authors themselves treat idea-centred scaffolding as most useful in this sample, not as a universal ranking.
Related
Claim ledger
What the evidence shows
Drawn from 6 studies in this library. Mix labels say which citation roles are present; they are not a strength score. Supports means evidence for a finding; Challenges means evidence against a stated position; Qualifies marks scope.
In a Hong Kong flipped education course, a historical-cohort quasi-experiment compared WeChat mobile instant messaging with asynchronous online discussion. Twenty-six MIM and 28 AOD students completed the quiz; interviews supplemented behavioural, emotional and cognitive engagement indicators. MIM better facilitated interaction and, on optional tasks, more idea exchange and creating — stronger cognitive engagement when participation was voluntary. Cohorts were separated in time, so semester differences could confound the app contrast.
Sixty-six Taiwanese sixth-graders (35 experimental, 31 control, from 89 enrolled) were randomly assigned after exclusions to a dialogue ITS or traditional remedial teaching. The ITS group outperformed controls on the post-test after prior-score adjustment (F = 5.52); pretest α = 0.879. Random assignment here is still a short remedial session in one regional cohort, not a year-long curriculum RCT.
Universidad Francisco de Vitoria evaluated faculty training as a single-group quasi-experiment with 695 participants. Attitudes toward innovation and the LMS improved, and professors reported greater knowledge of innovative teaching elements, with broadly similar full-time and part-time patterns. No control group means maturation, self-selection and campus change remain live; student learning was not the outcome.
In two all-female high schools in Kuwait’s Al-Jahra district, one school’s teachers received Digital Education Shifting training (experimental school 746 students; control school 667). Experimental-school students recorded higher engagement in digital instructional environments. School-level assignment confounds training with everything else that differs between those two schools; achievement transfer was not established.
An eight-week graduate Distance and Online Education course compared minimally guided, task-oriented and idea-oriented scaffolds for triad collaborative problem-solving. Ten volunteers joined; one withdrew, leaving about nine analysed participants. Idea-centred scaffolding strengthened links among idea contribution, metacognitive regulation and knowledge-artefact behaviours in this sample. That n makes the ranking exploratory, not a definitive scaffold league table.
Sixty-two graduate students in 15 groups (four to five each) produced discussion logs over eight weeks; AI-supported prediction and SNA (density F = 4.90) plus 62 reflections were used to flag at-risk collaborators. Analytic comparisons of network metrics are not a multi-institution randomised trial of dashboards, and prediction is not a guarantee of later achievement.
Debates
Tensions and limits
Some items are genuine disagreements on the same question. Others mark different assays, populations, or outcomes.
Unit of assignment and what 'worked' do not match. DES assigns schools (746 vs 667 students) and reports engagement. MIM assigns historical cohorts (26 vs 28 quiz completers) and reports interaction on optional tasks. The ITS paper randomises individuals after exclusions and reports an adjusted post-test. Faculty training has no comparison group. Scaffolding compares conditions among ~9 volunteers. These are not six estimates of one quasi-experimental engagement effect.
Unit of assignment and what 'worked' do not match. DES assigns schools (746 vs 667 students) and reports engagement. MIM assigns historical cohorts (26 vs 28 quiz completers) and reports interaction on optional tasks. The ITS paper randomises individuals after exclusions and reports an adjusted post-test. Faculty training has no comparison group. Scaffolding compares conditions among ~9 volunteers. These are not six estimates of one quasi-experimental engagement effect.
- Training teachers with DES boosts student engagement
- Mobile messaging can deepen course discussion engagement
- Can a dialogue ITS fix fraction multiplication?
- Faculty training that shifts teaching innovation attitudes
- Which scaffolding helps online collaborative problem-solving?
Study Role Design N Population Outcome Training teachers with DES boosts student engagement Supports OtherQuasi-experimental comparison of schools whose teachers received DES training vs control N=1413 · Experimental school 746 and control school 667 female high-school students Female high-school students in two Al-Jahra, Kuwait schools Student learning outcomes associated with Digital Education Shifting teacher training Mobile messaging can deepen course discussion engagement Supports OtherComparison of WeChat MIM vs asynchronous online discussion in a flipped course N=54 · 26 MIM and 28 AOD students completed the quiz Hong Kong flipped-course students using MIM or forum discussion Behavioral, cognitive, and emotional engagement under MIM vs AOD Can a dialogue ITS fix fraction multiplication? Supports OtherQuasi-experiment of dialogue-based fraction ITS vs traditional remedial instruction N=66 · 66 sixth graders analysed (35 ITS, 31 control) after exclusions from 89 enrolled Sixth-grade students in central Taiwan learning fraction operations Fraction multiplication/division achievement with ITS vs traditional remediation Faculty training that shifts teaching innovation attitudes Supports OtherPre–post faculty development program evaluating attitudes toward teaching innovation N=695 · 695 university professors participated University faculty (full-time and part-time) in a faculty training program Attitudes toward transformative/innovative teaching after the development experience Which scaffolding helps online collaborative problem-solving? Supports OtherSmall graduate-course comparison of CPS processes under three scaffolding types N=9 · 10 volunteers began; one withdrew mid-semester Chinese graduate students in a Distance and Online Education course Collaborative problem-solving process quality under idea-centered vs other scaffolds
PaperFren reads this as a limit on how far one study travels — different assays, populations, or outcomes — not a forced fight between papers.
Outcomes disagree in kind. ITS moves a fraction post-test. DES and MIM target engagement/interaction. Faculty training moves self-reported attitudes. AI-analytics moves SNA metrics and offers a decision aid. Importing 'the quasi-experiment worked' across those endpoints overclaims.
Outcomes disagree in kind. ITS moves a fraction post-test. DES and MIM target engagement/interaction. Faculty training moves self-reported attitudes. AI-analytics moves SNA metrics and offers a decision aid. Importing 'the quasi-experiment worked' across those endpoints overclaims.
- Can a dialogue ITS fix fraction multiplication?
- Training teachers with DES boosts student engagement
- Faculty training that shifts teaching innovation attitudes
- Can AI prediction dashboards help online groups?
Study Role Design N Population Outcome Can a dialogue ITS fix fraction multiplication? Supports OtherQuasi-experiment of dialogue-based fraction ITS vs traditional remedial instruction N=66 · 66 sixth graders analysed (35 ITS, 31 control) after exclusions from 89 enrolled Sixth-grade students in central Taiwan learning fraction operations Fraction multiplication/division achievement with ITS vs traditional remediation Training teachers with DES boosts student engagement Supports OtherQuasi-experimental comparison of schools whose teachers received DES training vs control N=1413 · Experimental school 746 and control school 667 female high-school students Female high-school students in two Al-Jahra, Kuwait schools Student learning outcomes associated with Digital Education Shifting teacher training Faculty training that shifts teaching innovation attitudes Supports OtherPre–post faculty development program evaluating attitudes toward teaching innovation N=695 · 695 university professors participated University faculty (full-time and part-time) in a faculty training program Attitudes toward transformative/innovative teaching after the development experience Can AI prediction dashboards help online groups? Supports OtherAI performance prediction and learning analytics applied to online graduate group work N=62 · 62 graduate students in 15 groups over an 8-week course Online graduate engineering students in collaborative writing groups Collaboration patterns and reflections under AI-supported performance prediction
PaperFren reads this as a limit on how far one study travels — different assays, populations, or outcomes — not a forced fight between papers.
Timeline
How understanding moved
Study years are when the paper was published. Evidence edits are dated changes to this page's claims. Explanations are when PaperFren added a Discovery — not a claim that the science happened that day.
2026
Concept page published
Quasi-Experimental Design
Change log
What changed
Dated edits to this page's evidence: studies added or removed from a claim, claims added or withdrawn, and new explanations tagged here. Coverage edits are distinct from reviewed research changes and corrections. Dates record PaperFren updates; study publication dates appear in the timeline.
- Coverage updated · Concept page published
Papers
6 studies in this library bear on Quasi-Experimental Design, ordered by citations.
- Can AI prediction dashboards help online groups?
Combining AI performance prediction with learning analytics reshaped analysis of online graduate groups' collaboration in an engineering course.
- Mobile messaging can deepen course discussion engagement
Compared with forum-style discussion, WeChat-based mobile instant messaging better supported interaction, relationships, and higher-level cognitive engagement in optional tasks.
- Training teachers with DES boosts student engagement
Kuwaiti high-school students whose teachers were trained on a Digital Education Shifting model reported higher engagement than students of untrained teachers.
- Which scaffolding helps online collaborative problem-solving?
In a small Chinese graduate course, idea-centered scaffolding best strengthened connections among ideas, metacognitive regulation, and knowledge-artifact work during CPS.
- Can a dialogue ITS fix fraction multiplication?
Sixth graders using a dialogue-based math ITS outperformed peers who received traditional remedial instruction on fraction operations.
- Faculty training that shifts teaching innovation attitudes
A university faculty development program improved professors’ attitudes toward teaching innovation and LMS use, plus knowledge of innovative teaching elements.
Compare studies
Select 2–10 studies. Design and N are labels, not a ranking.
Nothing selected yet.
Questions
What is still open
Unit of assignment and what 'worked' do not match. DES assigns schools (746 vs 667 students) and reports engagement. MIM assigns historical cohorts (26 vs 28 quiz completers) and reports interaction on optional tasks. The ITS paper randomises individuals after exclusions and reports an adjusted post-test. Faculty training has no comparison group. Scaffolding compares conditions among ~9 volunteers. These are not six estimates of one quasi-experimental engagement effect.
Outcomes disagree in kind. ITS moves a fraction post-test. DES and MIM target engagement/interaction. Faculty training moves self-reported attitudes. AI-analytics moves SNA metrics and offers a decision aid. Importing 'the quasi-experiment worked' across those endpoints overclaims.
Ask PaperFren about Quasi-Experimental Design
Study this conceptflashcards and short-answer questions
Why does training teachers in one Kuwaiti girls’ school (746 students) versus leaving another school as control (667) block a clean claim that DES caused higher engagement?
The assigned unit is the school. Any school difference — staffing, prior digital practice, student body — is confounded with DES. Engagement, not achievement, was the outcome, and the sample is two all-female schools in one district.
MIM showed more idea exchange on optional tasks than AOD. What design feature should appear in a limitation sentence?
Cohorts were sequential (historical quasi-experiment), n completing the quiz was 26 versus 28, and the course used WeChat. Semester events, not only the app, could produce the optional-task difference; mandatory high-stakes discussion might not look the same.
Faculty attitudes improved among 695 trainees with no control group. Name two non-training explanations for the pre/post rise.
Maturation or repeated testing on the same attitude instruments; self-selection into the programme; concurrent campus LMS or innovation initiatives. Student learning was not measured, so even a real attitude shift would not demonstrate better teaching outcomes.
How should you distinguish the ITS paper’s random assignment from calling the AI-analytics course a quasi-experiment of dashboards?
The ITS paper assigns students to tutoring versus traditional remediation and compares adjusted post-tests. The analytics paper applies prediction and SNA to 62 students’ existing group work (15 groups) and compares network metrics; there is no randomised dashboard-off condition. Analytic comparison ≠ assigned teaching condition.