Bias and fairness
Are dementia-prediction models less accurate for poorer patients?
Open access · cc by · source: Europe PMC
Standard machine-learning models predicting dementia from medical records made more errors for people of lower socioeconomic status, and rebalancing the training data by status narrowed but did not simply remove that gap.
Study at a glance
- Design
- Computational / modelling — Retrospective EHR-based modelling: LR, SVM, random forest and naive Bayes trained on 5 years of diagnosis codes to predict dementia in the next year, with bias measured by balanced error rate across SES-sex subgroups before and after SES oversampling.
- N
- Two cohorts, not one analytic N: the Mayo Clinic Study of Aging (N=5890 enrolled) and the Rochester Epidemiology Project (N=290,528 records); analysed samples after age matching and exclusions are smaller, split 70/30 train/test.
- Population
- Adults aged 50 and over in Olmsted County, Minnesota, from a research cohort and a population medical-records cohort
- Outcome
- Balanced error rate of dementia prediction within subgroups defined by SES (national ADI, state ADI, HOUSES Index) and sex
Structured fields used in claim comparison tables when every cited study has a complete layer.
Key findings
Across both cohorts and nearly all models, low-SES patients had higher balanced error rates than high-SES patients, the one exception being naive Bayes in the larger records cohort; high-SES men usually had the lowest error in the research cohort. The three SES measures overlapped poorly, for example only 12.0% of HOUSES-low patients were also low by national ADI. Oversampling on a given SES measure narrowed the gap on that measure, sometimes flipping it so low-SES patients did better, but could widen gaps on another measure (in one random forest case from 8.38% to 30.37%) and slightly reduced overall accuracy.
Methodology
Using two Minnesota cohorts, the authors built logistic regression, support vector machine, random forest and naive Bayes models that used five years of grouped diagnosis codes, age and sex to predict a dementia diagnosis in the following year. They measured fairness with the balanced error rate (the average of false-positive and false-negative rates) in subgroups defined by sex and three socioeconomic measures: national and state Area Deprivation Index and a housing-based HOUSES Index. They then oversampled low-SES patients in the training data with SMOTE-NC and retrained.
Limitations
Both cohorts come from one mostly White Midwestern county, so the size and direction of bias may differ elsewhere. Dementia in the records cohort was identified from diagnosis codes, and poorer patients are more likely to go undiagnosed, so some of the extra error may be label noise rather than model failure. The models used only coded diagnoses, not cognitive tests or biomarkers, so they are weak predictors built to study bias, not clinical tools. The authors did not formally test whether the oversampling improvements were statistically significant, and they used only one fairness metric.
How this study connects
Role on claims
Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.
Performance gaps by SES appeared consistently in these cohorts.
Models can err more for lower-socioeconomic patients: across both cohorts and nearly all models, low-SES patients had higher balanced error rates than high-SES patients.
Evidence for the claim as stated.
Mitigation moves bias around unless you check multiple measures.
Fixes targeted at one group definition can shift bias elsewhere: oversampling on one SES measure narrowed that gap but could widen gaps on another (8.38% to 30.37% in one case); in surgical assessment, baseline methods reduced underskilling but worsened overskilling, while TWIX modestly improved both.
Evidence for the claim as stated.
Group definitions don't agree: three SES measures overlapped poorly (only 12.0% of one measure's low-SES patients were low by another), so 'the' fairness gap depends on which measure is chosen.
Evidence for the claim as stated.
Open questions
Tensions this paper is part of
From concept pages' “where studies disagree.” Disagreement means the same question; scope means different assays, populations, or outcomes.
Group definitions don't agree: three SES measures overlapped poorly (only 12.0% of one measure's low-SES patients were low by another), so 'the' fairness gap depends on which measure is chosen.
Discoveries this paper informs or conflicts with
- Dropping race from a model can make it less fair, and fixing one gap can widen another
This paper informs this development.
Related papers in this topic
Same topic cluster — not a recommendation engine.