Topic · Mechanisms

Convolutions

Everything on this site about convolutions, in the order to read it: a page is listed after the pages it builds on. The figures below are the ones those pages use to explain it; each is live: move a slider and it recomputes.

In reading order

  1. Read · read first: DeiT — Training a Vision Transformer on ImageNet Alone, Swin Transformer — Windows, Shifts and a Pyramid for Dense Prediction
  2. Paper
  3. Paper · read first: Object Detection — A Quick Read
  4. Paper · read first: Understanding Attention Modules: CBAM and BAM — A Quick Read

The figures these pages use

Pulled from the pages above. Move a slider and the figure recomputes, exactly as on its page.

From Convolutions Strike Back — Was It the Attention or the Recipe? › 4. ConvNeXt: Take Swin's Design, Remove the Attention, Keep the Score

137
ResNet-50 to ConvNeXt-T, one step at a time. Top-1 on ImageNet-1k after each step of the ConvNeXt roadmap (Table 10 of the paper, three seeds), against Swin-T trained the same way. Step 1 touches only the training; everything after it is the network.

From Convolutions Strike Back — Was It the Attention or the Recipe? › 6. The Scoreboard

10
Four hybrids, three compute tiers. ImageNet-1k only, 224 px, 300 epochs, no distillation; each paper's own table (PVT Table 2, CvT Table 3, CoAtNet Table 4, ConvNeXt Table 1; references from DeiT Table 5, Swin Table 1, ConvNeXt Table 10). Top-1 is drawn as points above ResNet-50's original 76.1 %; R50 new is the same network with the DeiT/Swin recipe. Hollow bars are the reference models. Recipes and FLOP counters differ between papers, so read tiers, not decimals.

From MixConv — Mixed Depthwise Convolutional Kernels from Google Brain › MixConv — A new paradigm of Depthwise Separable Convolutional Kernels

1284
What mixing kernels costs. The mixed layer sees large and small patterns at once for far fewer weights than using the largest kernel everywhere.

From DetectoRS — A Comprehensive Review › Proposed Workflow

800
Resolution at each pyramid level. Side of the feature map at each level for the chosen input. f_i mixes the coarse, semantic map from above with the fine map x_i beside it.

From DetectoRS — A Comprehensive Review › Proposed Workflow

2
Cost of unrolling the pyramid T times. Backbone and FPN passes per image for each T. With shared weights the parameters stay the same; only compute grows.

From DetectoRS — A Comprehensive Review › Proposed Workflow

0.5
What one SAC output looks at. The nine weights land on the centre 3×3 (rate 1) and on the 7×7 corners and edges (rate 3). S decides how much of each.

From DetectoRS — A Comprehensive Review › Proposed Workflow

3
Atrous convolution. Filled cells are the nine pixels a 3×3 kernel reads at this rate. The outlined cell is the centre.

Tags that go with it

Efficient networks 2Attention modules 1Backbone design 1Feature pyramids and necks 1Object detection 1Training recipes 1Vision transformers 1