Topic · Mechanisms

Backbone design

Everything on this site about backbone design, in the order to read it: a page is listed after the pages it builds on. The figures below are the ones those pages use to explain it; each is live: move a slider and it recomputes.

In reading order

  1. Paper · read first: Object Detection — A Quick Read
  2. Paper · read first: YOLOv4 — Version 2: Bag of Specials
  3. Read · read first: DeiT — Training a Vision Transformer on ImageNet Alone, Swin Transformer — Windows, Shifts and a Pyramid for Dense Prediction
  4. Paper · read first: YOLOv4 — Version 2: Bag of Specials
  5. Paper
  6. Paper · read first: Object Detection — A Quick Read
  7. Paper
  8. Paper

The figures these pages use

Pulled from the pages above. Move a slider and the figure recomputes, exactly as on its page.

From YOLOv4 — Version 0: Introduction › 3. Performance Optimizations

100050
Cosine annealing with burn-in. The rate climbs through the burn-in, then follows half a cosine down to its floor.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 2. Bottleneck Attention Modules(BAM)

1.0
F' = F + F ⊗ M(F) on a feature map. An 8×8 feature map with two blobs, an attention map that favours the right one, and the result. Attention can only amplify.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 2. Bottleneck Attention Modules(BAM)

1.0-2.0
σ(M_c + M_s). The two branch scores add before the sigmoid, so either branch can push a position open on its own, and both together saturate.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 2. Bottleneck Attention Modules(BAM)

512
Cost of channel attention. A larger reduction ratio makes the attention module cheaper, at the price of a narrower bottleneck.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 2. Bottleneck Attention Modules(BAM)

2
What the spatial branch can see. Side of the window one output pixel sees, for each dilation value. Dilation widens the view without adding weights.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 3. Convolutional Block Attention Module(CBAM)

0.80.8
Channel attention, then spatial attention. Rows are channels, columns are positions. M_c rescales whole rows, M_s rescales whole columns. The product of the two gates is what survives.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 3. Convolutional Block Attention Module(CBAM)

0.9
Average pool and max pool see different channels. Eight channels of one feature map. Channel 6 is quiet everywhere except one spike: the average barely notices, the max does.

From Understanding Attention Modules: CBAM and BAM — A Quick Read › 3. Convolutional Block Attention Module(CBAM)

7
Where to look. Channel-wise average and max maps of a 12×12 feature, and the gate a k×k filter produces from them.

From Convolutions Strike Back — Was It the Attention or the Recipe? › 4. ConvNeXt: Take Swin's Design, Remove the Attention, Keep the Score

137
ResNet-50 to ConvNeXt-T, one step at a time. Top-1 on ImageNet-1k after each step of the ConvNeXt roadmap (Table 10 of the paper, three seeds), against Swin-T trained the same way. Step 1 touches only the training; everything after it is the network.

From Convolutions Strike Back — Was It the Attention or the Recipe? › 6. The Scoreboard

10
Four hybrids, three compute tiers. ImageNet-1k only, 224 px, 300 epochs, no distillation; each paper's own table (PVT Table 2, CvT Table 3, CoAtNet Table 4, ConvNeXt Table 1; references from DeiT Table 5, Swin Table 1, ConvNeXt Table 10). Top-1 is drawn as points above ResNet-50's original 76.1 %; R50 new is the same network with the DeiT/Swin recipe. Hollow bars are the reference models. Recipes and FLOP counters differ between papers, so read tiers, not decimals.

From Simple, Powerful, and Fast— RegNet Architecture from Facebook AI › Step 3. Making of RegNet Models — RegNetX and RegNetY

24362.5
From a straight line to stages. The line u_j rises steadily; rounding s_j turns it into a staircase, and each step of the staircase is one stage of the network.

From Simple, Powerful, and Fast— RegNet Architecture from Facebook AI › Step 3. Making of RegNet Models — RegNetX and RegNetY

24362.513
RegNet widths. One bar per block. Blocks of equal width form a stage; the line u_j is quantized into steps.

Tags that go with it

Object detection 3Feature pyramids and necks 2Image classification 2Training recipes 2Attention modules 1Convolutions 1Data augmentation 1Vision transformers 1