Topic · Mechanisms

Feature pyramids and necks

Everything on this site about feature pyramids and necks, in the order to read it: a page is listed after the pages it builds on. The figures below are the ones those pages use to explain it; each is live: move a slider and it recomputes.

In reading order

  1. Paper · read first: YOLOv4 — Version 1: Bag of Freebies
  2. Paper · read first: Attention Meets Pixels — Windows, Axes and the First Fully Attentional Networks, An Image Is Worth 16×16 Words — ViT, Patch by Patch, DeiT — Training a Vision Transformer on ImageNet Alone
  3. Paper · read first: Object Detection — A Quick Read
  4. Paper
  5. Paper · read first: Object Detection — A Quick Read
  6. Paper · read first: Object Detection — A Quick Read

The figures these pages use

Pulled from the pages above. Move a slider and the figure recomputes, exactly as on its page.

From YOLOv4 — Version 2: Bag of Specials › 1. Mish Activation

-1.0
Graph of the ReLU function. ReLU is a ramp that starts at 0. Left of 0 both the output and the slope are zero.

From YOLOv4 — Version 2: Bag of Specials › 1. Mish Activation

-1.2
Graph of the Mish function. Mish dips a little below zero for negative inputs instead of cutting them off, then follows the input.

From YOLOv4 — Version 2: Bag of Specials › 1. Mish Activation

-1.0
Mish against ReLU and Swish. Mish is smooth everywhere and lets small negative values through instead of cutting them to zero.

From YOLOv4 — Version 2: Bag of Specials › 2. CSP(Cross Stage Partial Connections)

166
What each dense layer reads. Input channels seen by each layer of a dense block. The same early channels (and their gradients) are reused by every later layer, which is the duplication CSP removes.

From YOLOv4 — Version 2: Bag of Specials › 6. DIoU NMS⁹

140.5
Keep or suppress, by DIoU. The dashed box M is the one already kept. Slide the candidate B_i across it: ordinary NMS looks only at IoU, DIoU-NMS also asks how far apart the centres are.

From Swin Transformer — Windows, Shifts and a Pyramid for Dense Prediction › 2. Window Attention and the Cost of Looking Everywhere

2247
Global attention against window attention, stage 1. Multiply-adds of one attention layer in stage 1 of Swin-T (C = 96, 4 × 4-pixel tokens) as the input side grows, on a log scale. The dotted line is the Q, K, V and output projections, which both versions pay. At 224 px, Eq. 1 and Eq. 2 of the paper give 2.00 G against 0.145 G per layer.

From Swin Transformer — Windows, Shifts and a Pyramid for Dense Prediction › 3. Shifted Windows

82
Windows, shifted windows, and the cyclic shift. Left: the regular partition. Middle: the same map with the partition moved by s tokens, which leaves small windows all round the edges. Right: the map rolled up and left by s so the regular partition fits again; the darker strips are tokens that wrapped around, and the mask stops them from talking to their new neighbours. Table 4: the shift is worth +1.1 top-1 and +2.8 box AP; Table 5: cyclic 3.0 ms against padding 3.3 ms in stage 1.

From DetectoRS — A Comprehensive Review › Proposed Workflow

800
Resolution at each pyramid level. Side of the feature map at each level for the chosen input. f_i mixes the coarse, semantic map from above with the fine map x_i beside it.

From DetectoRS — A Comprehensive Review › Proposed Workflow

2
Cost of unrolling the pyramid T times. Backbone and FPN passes per image for each T. With shared weights the parameters stay the same; only compute grows.

From DetectoRS — A Comprehensive Review › Proposed Workflow

0.5
What one SAC output looks at. The nine weights land on the centre 3×3 (rate 1) and on the 7×7 corners and edges (rate 3). S decides how much of each.

From DetectoRS — A Comprehensive Review › Proposed Workflow

3
Atrous convolution. Filled cells are the nine pixels a 3×3 kernel reads at this rate. The outlined cell is the centre.

Tags that go with it

Object detection 5Backbone design 2Activation functions 1Attention modules 1Convolutions 1Efficient attention 1Efficient networks 1Vision transformers 1