Can expert hints make surgical-skill AI fairer?
An AI that grades surgeons from video judged some groups more harshly than others, and teaching it which video moments experts consider important reduced those gaps while also making it more accurate.
Source
Human visual explanations mitigate bias in AI-based assessment of surgeon skills
Study at a glance
- Design
- Computational / modelling — Retrospective evaluation of a video-based surgical skill classifier (trained at one hospital, deployed at two others) for sub-cohort NPV/PPV gaps, before and after adding an expert frame-importance training signal (TWIX), compared with extra-data and video-pretraining baselines.
- N
- No single N reported in the main text; video-sample counts per hospital are in a table not reproduced. Metrics are averaged over ten Monte Carlo cross-validation folds.
- Population
- Video clips of needle handling and needle driving during robot-assisted prostatectomy from three hospitals, plus clips of medical students suturing on a model
- Outcome
- Negative and positive predictive value per surgeon sub-cohort (underskilling and overskilling bias), and overall AUC
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors examined SAIS, an AI system that labels short surgical video clips as low or high skill, across three hospitals and two suturing skills. They measured bias as differences in how often its low-skill verdicts (underskilling) or high-skill verdicts (overskilling) were correct for sub-groups such as novice versus expert surgeons or smaller versus larger prostates. They then retrained it with TWIX, which also asks the model to predict which frames human experts marked as important, and compared this with two standard mitigation methods.
What they found
The system showed underskilling and overskilling bias at every hospital, and which group was disadvantaged flipped depending on the skill and hospital, so checking one system at one site gave misleading conclusions. TWIX raised worst-case performance for disadvantaged groups, by up to 32% for needle driving at the training hospital, and improved overall AUC from 0.821 to 0.843 there. The baseline methods reduced underskilling more but made overskilling much worse, whereas TWIX modestly improved both. Gains were smaller or occasionally negative at other hospitals.
The limits
What it doesn't show
Bias is defined purely as any performance gap between sub-cohorts, with no threshold for what gap matters, and was measured once on retrospective data rather than in real credentialing. Groups such as race and sex of surgeons were excluded because of small samples, and only two suturing skills were studied. The mitigation effect varied by hospital and sometimes reversed, so TWIX is not shown to be a general fix.
Key terms
- Algorithmic bias
- Here, a difference in an AI system's performance between sub-groups of the people it assesses.
- Underskilling
- The AI wrongly rating a performance as low skill; tracked through negative predictive value.
- Overskilling
- The AI wrongly rating a performance as high skill; tracked through positive predictive value.
- Negative predictive value (NPV)
- The share of the model's low-skill predictions that are actually low skill.
- TWIX (training with explanations)
- Adding a training target that makes the model predict which video frames human experts flagged as important.
- AUC
- Area under the ROC curve, an overall measure of how well a classifier separates two classes.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
In this study, underskilling bias is measured as a gap in:
Common questions
Why does checking only one hospital or skill mislead?
The disadvantaged group flipped between skills and sites, so a single check could blame the wrong group or miss a bias entirely.
Did fixing bias cost accuracy?
Not here: TWIX improved overall AUC at the training hospital while shrinking gaps, though effects were weaker elsewhere.
Why not just use the baseline methods, which cut underskilling more?
They made the opposite error, overskilling, much worse, so the net fairness gain was small or negated.
More on Bias and fairness