Can ImageNet-trained networks read cancer tissue slides?
Features borrowed from a network trained on everyday photos, applied to small patches of huge tissue slides, classified and outlined cancer far better than hand-designed image features.
Source
Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features
Study at a glance
- Design
- Computational / modelling — AlexNet features from ImageNet extracted on image patches, pooled per slide and classified with a linear SVM; compared with hand-crafted features, whole-image CNN input, fine-tuning and prior methods
- N
- Several datasets: a brain-tumour challenge set (22 low-grade and 23 glioblastoma training slides for classification, 35 for segmentation) and a colon-cancer set of 717 cropped regions; no single N
- Population
- H&E-stained histopathology images of brain tumours (glioblastoma vs low-grade glioma) and colon cancer subtypes
- Outcome
- Classification accuracy and overlap-based segmentation accuracy
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
Whole tissue slides are far too large to feed into a neural network, so the authors cut each slide into overlapping patches, ran each patch through an AlexNet network pretrained on ImageNet, and pooled the patch features into one slide-level description for a linear SVM. For segmentation they classified small patches and let overlapping patches vote on each pixel. They tested this on a brain-tumour challenge dataset and a colon-cancer dataset, comparing against hand-crafted texture and colour features, resizing the whole image into the network, fine-tuning, and earlier published methods.
What they found
The approach won the MICCAI 2014 brain-tumour challenge with 97.5% classification accuracy and 84% segmentation accuracy on the hidden test set. Across both datasets the CNN features beat hand-crafted features by large margins (about 20% for brain-tumour classification), and patch sampling beat shrinking the whole slide by 8.2% in colon multiclass classification. Fine-tuning the network on histology gave only small extra gains, and heatmaps showed the classifier focused on regions pathologists consider diagnostic, such as necrosis.
The limits
What it doesn't show
The training sets are small (a few dozen brain slides), and because challenge test labels were withheld, most comparisons rely on cross-validation within the training data. The colon dataset comes from a single hospital, so generalisation to other labs and scanners is untested. The interpretive claims from heatmaps and neuron visualisations (for example a possible link between one subtype and haemorrhage) are qualitative observations, not validated findings, and the authors did not compare against fully convolutional segmentation networks.
Key terms
- Transfer learning
- Reusing a network trained on one large dataset (here ImageNet photos) as a feature extractor or starting point for a different task with little data.
- Patch sampling
- Cutting a very large image into many small overlapping tiles so each can be processed by a network with a fixed input size.
- Feature pooling
- Combining the feature vectors of many patches into one vector for the whole image, here with a 3-norm that emphasises strongly activated patches.
- Fine-tuning
- Continuing to train a pretrained network on the new task's data at a low learning rate so its features adapt to the new domain.
- Hand-crafted features
- Image descriptors designed by people, such as texture (LBP), gradient (SIFT) and colour histograms, rather than learned from data.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What was the main source of the image features used by the framework?
Common questions
Why not just shrink the whole slide and feed it to the network?
Slides can exceed 100,000 pixels on a side, so shrinking to network size destroys the cellular detail; the paper shows this whole-image approach does clearly worse than patch sampling.
Did training on tissue images (fine-tuning) matter a lot?
Not much here: it added roughly 0.4 to 1.6 percentage points in segmentation, far less than the jump from hand-crafted to pretrained CNN features.
Does this prove the network understands pathology?
No. Heatmaps show it attends to regions pathologists find meaningful, but that is a qualitative check, not proof of clinical validity.
More on Convolutional networks