Structured Receptive Fields in CNNs

Jörn-Henrik Jacobsen, Jan van Gemert, Zhongyou Lou, Arnold W.M. Smeulders

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

47 Citations (Scopus)


Learning powerful feature representations with CNNs is hard when training data are limited. Pre-training is one way to overcome this, but it requires large datasets sufficiently similar to the target domain. Another option is to design priors into the model, which can range from tuned hyperparameters to fully engineered representations like Scattering Networks. We combine these ideas into structured receptive field networks, a model which has a fixed filter basis and yet retains the flexibility of CNNs. This flexibility is achieved by expressing receptive fields in CNNs as a weighted sum over a fixed basis which is similar in spirit to Scattering Networks. The key difference is that we learn arbitrary effective filter sets from the basis rather than modeling the filters. This approach explicitly connects classical multiscale image analysis with general CNNs. With structured receptive field networks, we improve considerably over unstructured CNNs for small and medium dataset scenarios as well as over Scattering for large datasets. We validate our findings on ILSVRC2012, Cifar-10, Cifar-100 and MNIST. As a realistic small dataset example, we show state-of-the-art classification results on popular 3D MRI brain-disease datasets where pre-training is difficult due to a lack of large public datasets in a similar domain.
Original languageEnglish
Title of host publicationProceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
EditorsLisa O'Conner
Place of PublicationLos Alamitos, CA
Number of pages10
ISBN (Electronic)978-1-4673-8851-1
Publication statusPublished - 2016
EventCVPR 2016: 29th IEEE Conference on Computer Vision and Pattern Recognition - Las Vegas, United States
Duration: 26 Jun 20161 Jul 2016


ConferenceCVPR 2016
Country/TerritoryUnited States
CityLas Vegas


  • Scattering
  • Convolution
  • Kernel
  • Image resolution
  • Spatial coherence
  • Training data
  • Wavelet transforms


Dive into the research topics of 'Structured Receptive Fields in CNNs'. Together they form a unique fingerprint.

Cite this