Topic · Training

Data augmentation

Everything on this site about data augmentation, in the order to read it: a page is listed after the pages it builds on. The figures below are the ones those pages use to explain it; each is live: move a slider and it recomputes.

In reading order

  1. Read · read first: Object Detection — A Quick Read
  2. Paper · read first: YOLOv4 — Version 0: Introduction
  3. Paper · read first: An Image Is Worth 16×16 Words — ViT, Patch by Patch
  4. Paper · read first: YOLOv4 — Version 3: Proposed Workflow
  5. Paper · read first: Object Segmentation
  6. Paper · read first: Simple, Powerful, and Fast— RegNet Architecture from Facebook AI
  7. Project

The figures these pages use

Pulled from the pages above. Move a slider and the figure recomputes, exactly as on its page.

From DeiT — Training a Vision Transformer on ImageNet Alone › 3. The Recipe

00
The DeiT recipe, one ingredient at a time. Points of top-1 lost when one ingredient is taken out of the DeiT-B recipe (Table 8 of the paper). RA = RandAugment, Mix = Mixup, Cut = CutMix, neither = no Mixup and no CutMix, Rep = repeated augmentation, RE = random erasing, SD = stochastic depth. Hollow bars are runs that did not train at all with the fixed hyper-parameters.

From DeiT — Training a Vision Transformer on ImageNet Alone › 4. Distillation: Soft and Hard

0.73.0
Soft and hard distillation as the teacher's confidence changes. Two classes, true label A, the student fixed at p_s. The soft term is a bowl with its bottom at p_t = p_s; the hard term is a step at p_t = 0.5 and flat everywhere else. Table 3 of the paper: soft distillation 81.8 %, hard distillation 83.0 %.

Tags that go with it

Training recipes 4Object detection 3Backbone design 1Datasets and labelling 1Knowledge distillation 1Loss functions 1Segmentation 1Semi-supervised learning 1