Research method
Randomized Controlled Trial
3 studies
In this education library an RCT (or a close cousin) assigns classrooms or students by chance to a teaching condition and compares outcomes. Randomisation is at the unit the researchers can assign — a classroom’s emails, a CMC modality, a tutoring session — which may not be the unit a reader wants to generalise to. Quasi-experiments that randomise after exclusions still need to be read as small, local contrasts, not as proof that a whole curriculum works.
Education researchers reach for random assignment when they want to know whether a specific instructional move beats a control in that sample. It answers 'did this assignment change the measured outcome over this window?' Its main limitation here is scale: 15 classrooms with baseline imbalance, 40 EFL learners, 66 sixth-graders in a short remedial ITS — none is a multi-year, multi-institution curriculum trial.
Studies
3
Findings
3
3 supporting · 0 challenging · 0 qualifying citations
Open tensions
1
Currently
What we know
- A classroom-randomised similarity-email intervention in 505 biology students across 15 classrooms at 13 universities did not raise perceived similarity or mentoring quality in general. It did increase end-of-term similarity for students who started out feeling very different from the instructor. Intervention classrooms were larger at baseline, which the authors note generally dampens connection.
- Forty intermediate Iranian EFL learners randomised to text-based versus multimodal CMC for fourteen sessions split engagement by dimension: text-based was stronger on autonomous, behavioural and cognitive engagement; multimodal on emotional and social; satisfaction was similar. The window is fourteen sessions in one EFL context.
- A dialogue ITS for fraction multiplication and division used a quasi-experiment in which 66 Taiwanese sixth-graders were randomly assigned after exclusions. The experimental group outperformed controls on the posttest after adjusting for prior scores. The session was short remedial tutoring, not a semester-long rewrite, so durable classroom transfer is unproven.
Largest unresolved question
Unit of randomisation and what 'worked' disagree across the three papers. Classrooms were assigned in the mentoring study, so baseline enrolment size is a cluster confounder and the average treatment effect was null. Individuals were assigned in the CMC and ITS studies, which can show average gains in engagement or posttest scores without speaking to school-level implementation. Reading all three as 'RCTs of teaching' hides that.
Common misconceptions
If the average treatment effect is null, nobody benefited.
The similarity emails had no general effect on mentoring quality but did raise perceived similarity among students who began very different from the instructor. Heterogeneity is part of the result, not a failure of randomisation.
Randomising students to software for one remedial session shows the curriculum should be replaced.
The ITS result is a covariance-adjusted posttest advantage in a short session with 66 sixth-graders. Durable transfer and typical classroom control quality were not established.
The modality that produces more 'engagement' is the better writing treatment.
Text-based and multimodal CMC won different engagement dimensions while satisfaction stayed similar. Picking a winner requires saying which dimension and which writing outcome, which this fourteen-session study only partly measures.
Related
Claim ledger
What the evidence shows
Drawn from 3 studies in this library. Mix labels say which citation roles are present; they are not a strength score. Supports means evidence for a finding; Challenges means evidence against a stated position; Qualifies marks scope.
A classroom-randomised similarity-email intervention in 505 biology students across 15 classrooms at 13 universities did not raise perceived similarity or mentoring quality in general. It did increase end-of-term similarity for students who started out feeling very different from the instructor. Intervention classrooms were larger at baseline, which the authors note generally dampens connection.
Forty intermediate Iranian EFL learners randomised to text-based versus multimodal CMC for fourteen sessions split engagement by dimension: text-based was stronger on autonomous, behavioural and cognitive engagement; multimodal on emotional and social; satisfaction was similar. The window is fourteen sessions in one EFL context.
A dialogue ITS for fraction multiplication and division used a quasi-experiment in which 66 Taiwanese sixth-graders were randomly assigned after exclusions. The experimental group outperformed controls on the posttest after adjusting for prior scores. The session was short remedial tutoring, not a semester-long rewrite, so durable classroom transfer is unproven.
Debates
Tensions and limits
Some items are genuine disagreements on the same question. Others mark different assays, populations, or outcomes.
Unit of randomisation and what 'worked' disagree across the three papers. Classrooms were assigned in the mentoring study, so baseline enrolment size is a cluster confounder and the average treatment effect was null. Individuals were assigned in the CMC and ITS studies, which can show average gains in engagement or posttest scores without speaking to school-level implementation. Reading all three as 'RCTs of teaching' hides that.
Unit of randomisation and what 'worked' disagree across the three papers. Classrooms were assigned in the mentoring study, so baseline enrolment size is a cluster confounder and the average treatment effect was null. Individuals were assigned in the CMC and ITS studies, which can show average gains in engagement or posttest scores without speaking to school-level implementation. Reading all three as 'RCTs of teaching' hides that.
- Can showing shared interests improve student-teacher mentoring?
- Text or multimodal CMC: which engages EFL writers?
- Can a dialogue ITS fix fraction multiplication?
Study Role Design N Population Outcome Can showing shared interests improve student-teacher mentoring? Supports OtherCluster-randomized classroom trial emailing shared instructor–student similarities N=505 · 505 biology students in 15 classrooms across 13 US universities College biology students in semester-long research courses Perceived similarity and mentoring relationship quality, especially for initially dissimilar students Text or multimodal CMC: which engages EFL writers? Supports Human experimentRandom assignment to text-based vs multimodal CMC for 14 treatment sessions N=40 · 40 intermediate Iranian EFL learners (20 per condition) Intermediate EFL learners in CMC writing groups Autonomy, multi-dimensional engagement, satisfaction, and writing performance by modality Can a dialogue ITS fix fraction multiplication? Supports OtherQuasi-experiment of dialogue-based fraction ITS vs traditional remedial instruction N=66 · 66 sixth graders analysed (35 ITS, 31 control) after exclusions from 89 enrolled Sixth-grade students in central Taiwan learning fraction operations Fraction multiplication/division achievement with ITS vs traditional remediation
PaperFren reads this as a limit on how far one study travels — different assays, populations, or outcomes — not a forced fight between papers.
Timeline
How understanding moved
Study years are when the paper was published. Evidence edits are dated changes to this page's claims. Explanations are when PaperFren added a Discovery — not a claim that the science happened that day.
2026
Concept page published
Randomized Controlled Trial
Change log
What changed
Dated edits to this page's evidence: studies added or removed from a claim, claims added or withdrawn, and new explanations tagged here. Coverage edits are distinct from reviewed research changes and corrections. Dates record PaperFren updates; study publication dates appear in the timeline.
- Coverage updated · Concept page published
Papers
3 studies in this library bear on Randomized Controlled Trial, ordered by citations.
- Text or multimodal CMC: which engages EFL writers?
Text-based CMC boosted autonomy and cognitive/behavioral engagement, while multimodal CMC boosted emotional and social engagement.
- Can showing shared interests improve student-teacher mentoring?
Sharing common interests only helps students who initially feel very different from their instructors feel closer to them, which indirectly improves their mentoring relationship.
- Can a dialogue ITS fix fraction multiplication?
Sixth graders using a dialogue-based math ITS outperformed peers who received traditional remedial instruction on fraction operations.
Compare studies
Select 2–10 studies. Design and N are labels, not a ranking.
Nothing selected yet.
Questions
What is still open
Unit of randomisation and what 'worked' disagree across the three papers. Classrooms were assigned in the mentoring study, so baseline enrolment size is a cluster confounder and the average treatment effect was null. Individuals were assigned in the CMC and ITS studies, which can show average gains in engagement or posttest scores without speaking to school-level implementation. Reading all three as 'RCTs of teaching' hides that.
Ask PaperFren about Randomized Controlled Trial
Study this conceptflashcards and short-answer questions
Why does randomising classrooms rather than students make the mentoring-email trial harder to interpret?
The assigned unit is the classroom, so anything that differs by classroom — here, larger enrolment in intervention rooms — is a cluster-level confound. Student-level outcomes then mix the email treatment with class size, which the authors flag as dampening connection.
The ITS paper randomises after exclusions and adjusts for prior scores. What kind of design is that, and what does the posttest advantage still not show?
A small randomised (after exclusion) quasi-experiment. It shows a short-term adjusted posttest gain in one regional cohort, not that a year-long curriculum change would transfer or that every control classroom was equally well taught.
How can both CMC modalities 'win' without the trial being contradictory?
They win on different engagement subscales: text-based on autonomy/behavioural/cognitive, multimodal on emotional/social, with similar satisfaction. A unidimensional 'engagement' average would have hidden that split.
Compare this education RCT page with the medicine RCT page: what is alike in the logic of randomisation, and what is unlike in the outcomes?
Alike: chance assignment is meant to balance confounders so a later contrast can be attributed to the assigned condition. Unlike: medicine trials here often use validated symptom scores, retention figures, and sometimes non-inferiority margins; the education trials are small instructional contrasts (emails, 14 CMC sessions, one ITS sitting) without hard clinical endpoints. Importing medical 'the trial proved it' language overclaims these papers.