DETR — Detection as Set Prediction, No Anchors and No NMS

Every detector in Object Detection — A Quick Read predicts a set of boxes, but none of them is trained as one. [CARI20] put the problem statement in a sentence: detectors "address this set prediction task in an indirect way, by defining surrogate regression and classification problems on a large set of proposals, anchors, or window centers", and then lean on "postprocessing steps to collapse near-duplicate predictions". DETR (May 2020) removes the surrogates. The pipeline is one line: CNN → transformer encoder → transformer decoder fed N learned queries → N (class, box) pairs, matched one-to-one with the ground truth.

Brush up before you start ;)

• The encoder–decoder, multi-head attention, FFN, residual + LayerNorm and sinusoidal positions: Attention Is All You Need — The Transformer, Block by Block. I will only say what DETR changes.
• Anchors, proposals, NMS, IoU and the AP vocabulary: Object Detection — A Quick Read. Class imbalance and focal loss: Understanding Focal Loss —A Quick Read.
• Why a transformer over a feature map has to watch its token count: Attention Meets Pixels — Windows, Axes and the First Fully Attentional Networks.

1. Detection as Set Prediction

Picture 100 raised hands and 7 questions. Each question goes to the hand that already looks closest to the answer; everyone else puts their hand down (that is ∅). No two hands get the same question, so nobody needs shushing afterwards. That is DETR: the 100 hands are the object queries, the pairing is the bipartite matching, the shushing it does not need is NMS.

Two things make a prediction a set rather than a list:

• A fixed size N, larger than any image needs. N = 100; COCO averages 7 instances per image and tops out at 63 in train (§4). The spare slots predict a special class ∅, "no object", which "plays a similar role to the 'background' class in the standard object detection approaches" (§3.2).
• A loss that does not care about order. The matching of §3 pairs each ground truth with exactly one slot, so permuting the slots changes nothing. The paper's words: the matching "plays the same role as the heuristic assignment rules used to match proposals or anchors to ground truth objects in modern detectors. The main difference is that we need to find one-to-one matching for direct set prediction without duplicates" (§3.1).

2. The Architecture, Tensor by Tensor

Let's follow the shapes through Fig. 1.

• Backbone. $x_{img}\in {ℝ}^{3\times H_0\times W_0}$ → ResNet-50 with frozen BatchNorm → $f\in {ℝ}^{2048\times H\times W}$, $H=H_0/32$, $W=W_0/32$. A 1×1 convolution brings 2048 down to $d=256$; the map is flattened into $HW$ tokens of width 256.
• Encoder. Six standard layers of Attention Is All You Need — The Transformer, Block by Block, 8 heads. One change: fixed 2-D sine positional encodings are added to the queries and keys at every attention layer, not once at the input (A.3).
• Decoder. Six layers. The input is not a shifted target sequence but N = 100 object queries: learned embeddings in ${ℝ}^{100\times 256}$ (Listing 1 of the paper initialises them as nn.Parameter(torch.rand(100, hidden_dim))). They are decoded in parallel, with self-attention among the 100 and cross-attention to the HW encoder tokens. No causal mask, no autoregressive loop (§3.2).
• Heads. A 3-layer MLP (hidden 256, ReLU) gives $(c_x,c_y,w,h)\in [0,1{]}^4$ through a sigmoid; a linear layer gives a softmax over 91 COCO classes plus ∅. Output: 100 × (92 logits + 4 coordinates). The same heads, with auxiliary matching losses, sit after every decoder layer.

At the largest input the paper allows, shortest side 800 and longest side 1333 (§4), the encoder sees $800/32\times 1333/32=25\times 41=$ 1,025 tokens of width 256. The DC5 variant doubles the map's resolution: 50 × 82 = 4,100 tokens, so the encoder's attention matrix is 16× larger, which the paper books as "an overall 2x increase in computational cost" (§4). That is Eq. 1 of Attention Meets Pixels — Windows, Axes and the First Fully Attentional Networks deciding a design. Parameters: 41.3 M = 23.5 M ResNet-50 + 17.8 M transformer (§4.1).

NOTE: DETR does not remove the backbone. ResNet-50 stays; what the transformer replaces is the region proposal network, the RoI heads and NMS.

3. Bipartite Matching

3.1 The Problem

Call the ground truth $y=\{y_i\}$, padded with ∅ up to size N, and the predictions $\widehat{y}=\{{\widehat{y}}_i{\}}_{i=1}^N$. We want the pairing with the smallest total cost (Eq. 1 of the paper):

• $\sigma$: a permutation of N elements; ${𝔖}_N$ is the set of all N! of them; $\sigma (i)$ is the slot assigned to ground truth $i$.
• The Hungarian algorithm finds $\widehat{\sigma }$ in polynomial time. Nobody tries 100! permutations. My picture for it: a wedding seating plan, guests (truths), chairs (slots), a cost per pair, and you want the cheapest full seating, not the greedy one.

3.2 The Cost of One Pair

Each ground truth is $y_i=(c_i,b_i)$: a class $c_i\in \{1.\mathrm{.91}\}\cup \{\varnothing \}$ and a box $b_i\in [0,1{]}^4$ as centre, height and width relative to the image. The pair cost is (§3.1):

• ${\widehat{p}}_{\sigma (i)}(c_i)$: the slot's softmax probability for the true class; ${\widehat{b}}_{\sigma (i)}$: its box; $\mathbb{1}$: an indicator that switches both terms off for padded ∅ entries.
• A slot is cheap for an object if it already believes in that class and its box is close. The class term uses the probability, not its log, on purpose: the paper says it keeps the term commensurable with the box term.
• Padded rows cost the same whatever the slot does, so they never influence which slot gets a real object. With M real objects the matcher effectively works on an M × 100 matrix; for a typical COCO image that is 7 × 100 = 700 costs.

Now the same thing with numbers. The grid below is Eq. 2 for three objects and eight of the hundred slots; raise the class weight, or drag slot 4 across the image, and watch the outlined cells jump.

• At the paper's weight (1) the matcher gives object A to slot 7, whose box fits; from a class weight of 3.25 on it switches to slot 2, which is surer of class A (0.55 against 0.25) even though its box is worse.
• Slide slot 4 right: between a shift of 0.22 and 0.34 its box sits on object C and it steals it from slot 1, although slot 1 believes in C more (0.45 against 0.30); push further and slot 1 gets it back. The matcher weighs belief and geometry together, every time.

4. The Set Loss

4.1 The Hungarian Loss

With $\widehat{\sigma }$ fixed, the loss is an ordinary classification + box loss over every slot (Eq. 2 of the paper):

• Compare with Eq. 2: the probability became a log-probability, and the class term no longer has an indicator. Every one of the 100 slots gets a classification loss: the matched ones toward their object's class, the other ~93 toward ∅. Only matched slots get a box loss.
• No gradient flows through the matcher. $\widehat{\sigma }$ is a discrete assignment treated as a constant; only the loss terms are differentiated.
• Imbalance: in a 7-object image, 93 % of the class terms say "nothing here". The ∅ log term is down-weighted by 10, which the paper calls "analogous to how Faster R-CNN training procedure balances positive/negative proposals by subsampling" (§3.1). This is DETR's own trick; the focal-loss answer to the same problem is in Understanding Focal Loss —A Quick Read, and Deformable DETR will use it.
• Losses are normalised by the number of objects in the batch (A.2).

4.2 The Box Loss

ℓ1 on normalised coordinates has a scale problem: a 10-pixel miss is a disaster on a small box and a rounding error on a large one. So the box loss mixes ℓ1 with a scale-free term (Eq. 9–10 of the paper, weights from A.4):

• $\Vert ·{\Vert }_1$: the sum of absolute differences of the four coordinates; $|·|$: area; $B(b,\widehat{b})$: the smallest box enclosing both.
• The bracket is the generalised IoU (the paper's ref. [38]): IoU minus the empty fraction of the enclosing box. When the boxes stop overlapping, IoU is 0 and stuck, but the second term keeps growing, so there is still a gradient. GIoU is not "IoU with a sign"; it is IoU with a penalty for the air between the boxes.
• The same ${ℒ}_{\text{box}}$, with the same weights, is used inside Eq. 2 (§4.2: "using the same weighting every time").

Let's slide a prediction off its box and watch the three terms. In the figure the prediction keeps the truth's shape and moves along x; push it off vertically with dy, or change its size, and the overlap dies sooner.

• Past the dotted line the IoU loss is flat at 1 and would teach nothing; the GIoU loss keeps climbing toward 2. And the third tile is the scale problem: the same 0.4 shift gives the same ℓ1 on a 0.06 × 0.04 box, but a much larger GIoU loss, which is the whole reason for mixing the two.

5. What the Queries Learn

• Slots are not anchors and not class slots. Fig. 7 of the paper shows each slot specialising in regions and box sizes, with several modes each; "almost all slots have a mode of predicting large image-wide boxes".
• They generalise past the training counts. Fig. 5: all 24 giraffes in a synthetic image are found, although no training image has more than 13.
• Self-attention among the queries is the duplicate remover. Fig. 4: adding NMS (IoU 0.7) helps after decoder layer 1, because "there is no communication in the first layer", and slightly hurts after layer 6. Decoder depth is worth +8.2 AP from layer 1 to layer 6.
• Position matters, even for a set. Table 3: no spatial encoding at all costs 7.8 AP (32.8). The encoder tokens carry 2-D sines; the object queries are learned positional encodings.
• The decoder "typically attends to object extremities, such as legs and heads" (Fig. 6). Removing the encoder entirely costs 3.9 AP (Table 2).

6. Results

Table. 1 Table 1 of [CARI20], COCO val. FLOPS and FPS measured on torchscript models.

• The headline is same AP, better large, worse small: 42.0 against 42.0, AP_L 61.1 vs 53.4, AP_S 20.5 vs 26.6.
• Half the FLOPs (86 vs 180) and 28 FPS against 26. DETR is not slow at inference; it is slow to train: 500 epochs for Table 1, and 300 epochs take 3 days on 16 V100s (§4). Keep the two sentences apart.
• Ablations: class + GIoU only, 39.9 (−0.7); class + ℓ1 only, 35.8 (−4.8); all three, 40.6 (Table 4). Overriding a ∅ prediction with its second-best class at test time is worth +2 AP (§4).
• The conclusion names the open problems itself: "training, optimization and performances on small objects".

7. Slow Convergence and Its Fix

[ZHU20] (Deformable DETR, 2021 ICLR) start from the two problems above: "DETR needs 500 epochs to converge, which is around 10 to 20 times slower than Faster R-CNN" and has "relatively low performance at detecting small objects" (§1). Their diagnosis (§3.1) has two parts:

• Encoder self-attention is quadratic in HW, so high-resolution maps, the ones small objects need, are unaffordable (the 16× of §2).
• "At initialization, the cross-attention modules are almost of average attention on the whole feature maps", while at the end of training they are very sparse. Learning that sparsity from a uniform start is what takes 500 epochs.

The fix keeps the matching and the set loss and changes only what a query reads. Instead of a softmax over all HW keys, each query reads K sampled points per head (their Eq. 2):

• $z_q$: the query feature; $p_q$: its 2-D reference point; $x\in {ℝ}^{C\times H\times W}$; M = 8 heads; K = 4 samples per head; $\Delta p_{mqk}\in {ℝ}^2$: a learned offset; $A_{mqk}$: a softmaxed weight; $W_m,W_m^{\prime }$: the output and value projections. $x(·)$ is read by bilinear interpolation, so the offsets can be fractional.
• Offsets and weights both come from one linear layer on $z_q$ (3MK channels: 2MK offsets, MK weights). The multi-scale version samples over L = 4 feature levels.
• The bill: deformable attention costs $O(2N_q{C}^2+min(HW{C}^2,N_{q}K{C}^2))$ for $N_q$ queries (§4.1, A.1). In the encoder $N_q=HW$, so DETR's $O(H^2{W}^{2}C)$ self-attention becomes $O(HW{C}^2)$, linear in the map size, which is what lets the encoder read multi-scale maps at all.

NOTE: deformable attention is not a deformable convolution, although both sample at learned fractional offsets. A deformable conv applies a fixed kernel weight at each sample; here the weight A_mqk is predicted and softmax-normalised, and the query may be an object query rather than a pixel.

Table. 2 Training cost against accuracy, Table 1 of [ZHU20] (COCO val; the first three rows repeat Table. 1 above with the epochs and GPU-hours [ZHU20] add). Run time on a V100.

• 10× fewer epochs (50 vs 500) and 6× fewer GPU-hours (325 vs 2000) for +1.8 AP, and AP_S goes from 20.5 to 26.4, level with Faster R-CNN. DETR-DC5 at the same 50 epochs sits at 35.3; focal loss and 300 queries alone only buy 36.2.

8. What Came Next

Five months after DETR, An Image Is Worth 16×16 Words — ViT, Patch by Patch (October 2020) asked a stranger question: what if there is no CNN at all, and no decoder either? Cut the image into 16×16 patches, feed the 196 tokens to the plain encoder of Attention Is All You Need — The Transformer, Block by Block, and read the class off one extra token. DETR and ViT are independent 2020 papers; the cost arithmetic of §2 is also why a transformer backbone at detection resolution had to wait for the windows of Swin Transformer — Windows, Shifts and a Pyramid for Dense Prediction. Stay tuned.

9. References

1. [CARI20] Carion, Massa, Synnaeve, Usunier, Kirillov, Zagoruyko. End-to-End Object Detection with Transformers. 2020, arXiv 2005.12872.
2. [ZHU20] Zhu, Su, Lu, Li, Wang, Dai. Deformable DETR: Deformable Transformers for End-to-End Object Detection. ICLR 2021, arXiv 2010.04159.
3. [VASW17] Vaswani et al. Attention Is All You Need. 2017, arXiv 1706.03762.