Topic · Mechanisms

Loss functions

Everything on this site about loss functions, in the order to read it: a page is listed after the pages it builds on. The figures below are the ones those pages use to explain it; each is live: move a slider and it recomputes.

In reading order

  1. Read
  2. Read
  3. Paper · read first: YOLOv4 — Version 0: Introduction
  4. Paper
  5. Paper
  6. Paper

The figures these pages use

Pulled from the pages above. Move a slider and the figure recomputes, exactly as on its page.

From Object Detection — A Quick Read › Literature Survey

1.0
MSE, MAE and Huber loss. Huber follows the squared loss near zero and the absolute loss far from it, so one large error cannot dominate.

From Object Detection — A Quick Read › Literature Survey

0.5
Cross-entropy for y = 1 and y = 0. Left: a foreground example. Right: a background example. Even a confident, correct prediction still carries some loss.

From Object Detection — A Quick Read › Literature Survey

121.0
Intersection over Union. The dashed box is the ground truth, the solid box is the prediction, and the shaded area is their overlap.

From Understanding Focal Loss —A Quick Read › Focal Loss

0.8
From p to p_t. Whatever the label, p_t is 'how right the model was'. A large p_t is an easy example.

From Understanding Focal Loss —A Quick Read › Focal Loss

0.25
α-balanced cross-entropy. α scales the foreground curve and 1 − α the background curve. It changes how much each class counts, not which examples are hard.

From Understanding Focal Loss —A Quick Read › Focal Loss

2.0
The modulating factor (1 − p_t)^γ. γ = 0 is plain cross-entropy. As γ grows, everything to the right of p_t ≈ 0.5 is pushed towards zero.

From Understanding Focal Loss —A Quick Read › Focal Loss

4.00.25
Focal loss for y = 1 and y = 0. Left: a foreground example. Right: a background example. A larger γ flattens the side where the model is already right.

From Understanding Focal Loss —A Quick Read › Focal Loss

2.00.25
Focal loss and cross-entropy. Loss for a positive example as the model grows confident. The dashed line is cross-entropy.

Tags that go with it

Object detection 4Re-identification 2Class imbalance 1Data augmentation 1Faces 1Non-maximum suppression 1Training recipes 1