Can one neural network score sleep across any clinic's data?
A single network trained on very diverse sleep recordings scored sleep about as well as the best of five human experts, even on datasets it never saw, without retraining.
Source
U-Sleep: resilient high-frequency sleep staging
Study at a glance
- Design
- Computational / modelling — U-Net-style fully convolutional network trained on pooled polysomnography datasets with random EEG/EOG channel sampling; evaluated on held-out subjects, held-out datasets and consensus-scored data.
- N
- N=15660 · 15,660 participants contributing 19,924 PSG records across 21 datasets; consensus comparisons used 25 healthy subjects (DOD-H) and 55 OSA patients (DOD-O).
- Population
- Polysomnography recordings from healthy people and sleep-disordered patients in 16 clinical studies, mainly from North America and Europe.
- Outcome
- Per-stage and mean F1 agreement with expert sleep stage labels; F1 of a classifier separating OSA patients from controls using high-frequency stages.
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors trained U-Sleep, a fully convolutional encoder-decoder network inspired by U-Net, on 19,924 overnight recordings from 15,660 participants in 16 clinical studies. During training each example used a randomly chosen EEG and EOG channel pair, so the model had to work regardless of electrode placement, hardware or sampling rate. Eight datasets were kept entirely out of development to test generalisation, and two consensus-scored datasets let them compare the model with five clinical experts. They also tested whether the model's very high-frequency stage outputs could help a simple classifier separate sleep apnea patients from healthy people.
What they found
Across 21 datasets the mean F1 was 0.79, with stage N1 hardest to score (about 0.53) and Wake and REM easiest. On 25 healthy subjects the model matched the best human expert (F1 0.79), and on 55 apnea patients it scored 0.76, higher than every expert, though neither difference from the best expert was statistically significant. Majority voting across channel combinations was always at least as good as the average channel. Features from high-frequency stage outputs raised apnea-versus-healthy classification F1 from 0.60 to a peak of 0.94.
The limits
What it doesn't show
Almost all data came from Europe and North America and mostly from healthy people or apnea patients, so performance on people with highly abnormal brain activity (stroke, neurodegeneration, psychiatric illness) is unknown. The authors did not tune the architecture, and limited demographic information means some biases cannot be ruled out. The apnea classification with high-frequency stages is a small proof-of-concept on two datasets, not a validated diagnostic test. Ground truth labels are themselves noisy human scores, which caps how accuracy can be judged.
Key terms
- Polysomnography (PSG)
- An overnight recording of brain waves (EEG), eye movements (EOG), muscle activity and other signals used to diagnose sleep disorders.
- Sleep staging
- Labelling each segment of a night's recording as Wake, N1, N2, N3 or REM sleep.
- F1 score
- The harmonic mean of precision and recall; here it measures agreement between predicted and reference stage labels.
- Fully convolutional network
- A neural network made only of convolution-type layers, so it can process inputs of any length in one pass.
- Out-of-distribution evaluation
- Testing a model on data from sources never used in training, to see whether it generalises to new settings.
Flashcards
0 of 9 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What inputs does U-Sleep require?
Common questions
Why train on randomly chosen channels?
Clinics record different electrode sets; forcing the model to cope with any EEG and EOG pair makes it usable anywhere without retraining.
Does matching the best expert mean the model is always right?
No. It means its agreement with the expert consensus was similar to one expert's; human scoring itself disagrees, especially for stage N1.
What are high-frequency sleep stages?
The model can output a stage prediction many times per second instead of once per 30 seconds, capturing brief changes that standard scoring ignores.
More on Generalisation and overfitting