When do a CNN's first filters learn whole DNA motifs?
CNNs only learn whole, readable DNA motifs in their first layer when the architecture stops deeper layers from assembling motifs out of pieces.
Source
Representation learning of genomic sequence motifs with convolutional neural networks
Study at a glance
- Design
- Computational / modelling — Systematic architecture sweep of CNNs (varying max-pool size, filter size, filter count) on synthetic sequences with planted motifs, then on in vivo ChIP-seq sequences.
- N
- 25,000 synthetic sequences (split into train/validation/test); in vivo set of 270,382 training and 23,768 test sequences.
- Population
- Synthetic 200-nucleotide DNA sequences with 12 embedded transcription-factor motifs; human genomic sequences from the DeepSEA/ENCODE dataset
- Outcome
- Fraction of first-layer filters matching ground-truth motifs; classification AU-ROC
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
The authors planted known transcription-factor motifs into random DNA so the correct answer was known, then trained small convolutional networks that differed only in max-pool size, filter size or filter count. They compared each first-layer filter against the planted motifs to measure how many had learned a whole motif. They then repeated the comparison on real ChIP-seq sequences.
What they found
Networks with small first-layer max-pooling learned scattered partial motifs (at best a fraction of 0.127 of filters matched a true motif), while large pooling pushed nearly all filters to learn whole motifs (up to a fraction of 0.980 with a pool size of 25), even though classification accuracy was similar. When pooling was absent, the second layer assembled the whole motifs instead. Matching filters to a motif database was misleading, because many filters matched some database motif without matching the true one. On real data, the distributed-representation networks performed slightly better.
The limits
What it doesn't show
The synthetic task assumes binding is fully explained by a single motif-like pattern, which is much simpler than real regulatory DNA. The study only varies architecture and does not test how regularisation or optimisation choices shape representations. For real sequences there is no ground truth, so filter quality could only be compared to known motifs. Visualising a filter also says nothing about how much the network actually relies on it.
Key terms
- Max-pooling
- A layer that keeps only the largest activation in each window, shrinking the output and blurring exact positions.
- Localist representation
- A code where a single unit (here, one filter) stands for a whole pattern such as a complete motif.
- Distributed representation
- A code where a pattern is spread across several units, each capturing a part that later layers combine.
- Sequence motif
- A short, recurring DNA pattern, such as the site a transcription factor binds.
- AU-ROC
- Area under the ROC curve; a classification score where 0.5 is chance and 1 is perfect.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
What mainly determined whether first-layer filters learned whole motifs?
Common questions
Why use synthetic data instead of real genomes?
Because the planted motifs are known exactly, the authors could check whether each filter learned the right pattern, which is impossible with real data.
Does a better-interpretable network mean a better-performing one?
Not necessarily; accuracy was similar across designs on synthetic data, and on real data the less interpretable distributed designs did slightly better.
Why is comparing filters to a motif database unreliable?
Many filters matched some motif in the database while matching none of the true planted motifs, so a database hit does not prove the filter learned something relevant.
More on Interpretability