Skip to content
PaperFren

Representation learning

Can a network trained on everyday photos search pathology slides?

Hegde N, Hipp JD, Liu Y, et al. · NPJ digital medicine · 2019

Open access · cc by · source: Europe PMC

A neural network that had never seen tissue images could still find visually similar pathology patches far better than chance and better than a classic hand-crafted method.

Save to libraryView originalAsk PaperFrenStudy view

Study at a glance

Design
Computational / modelling — Image-retrieval benchmark on pathologist-annotated TCGA slides comparing a pretrained deep-ranking embedding with SIFT and random retrieval, plus a blinded rating study by pathologists.
N
Database of 127,000 patches from 45 slides; query set of 22,500 patches from 15 other slides; three pathologists rated a subsample.
Population
Digitised breast, prostate and colon tissue slides from The Cancer Genome Atlas.
Outcome
Top-5 retrieval accuracy for matching histologic feature, organ and Gleason pattern; pathologist match ratings.

Structured fields used in claim comparison tables when every cited study has a complete layer.

Key findings

For prostate queries, at least one of the top five results showed the same tissue feature 62.0% of the time, versus 44.2% for SIFT features and 28.3% for random retrieval. For prostate cancer grade, SMILY beat SIFT at 73.1% versus 62.1%. In the blinded study pathologists judged SMILY results as matching the tissue feature 62.1% of the time versus 26.8% for random results; matches were often right on tissue type but wrong on organ.

Methodology

The team used a deep ranking network, trained on about 500,000,000 everyday photos to place similar images close together, to turn small patches of pathology slides into 128-number embeddings. A query patch is matched to the nearest embeddings in a database built from 127,000 patches from 45 slides. They scored retrieval against pathologist annotations and also had three pathologists, blinded to the source, rate SMILY results against random results.

Limitations

Annotations were coarse and incomplete, so some correct matches were scored as errors, and only three pathologists rated results with no measurement of their agreement. The test used a modest number of slides from one public archive and only a few organs and similarity axes. A general-purpose search tool is likely less accurate than a model trained for one specific task, and the study does not show any effect on diagnoses.

How this study connects

Role on claims

Each row is a claim on a concept or method page where this paper supports, challenges, or qualifies the statement. Roles are hand-checked — not a model guess.

  • SupportsRepresentation learning and pretrainingconcept

    Learned embeddings beat hand-crafted descriptors for similarity search.

    A general-purpose image embedding (SMILY) returned a matching tissue feature in the top five 62.0% of the time for prostate queries, versus 44.2% for SIFT features and 28.3% for random retrieval.

    Evidence for the claim as stated.

Related papers in this topic

Same topic cluster — not a recommendation engine.