AutoFlow: Learning a Better Training Set for Optical Flow

Deqing Sun Daniel Vlasic Charles Herrmann Varun Jampani Michael Krainin Huiwen Chang
Ramin Zabih William T. Freeman Ce Liu
Google Research
| Paper | Samples | Code (coming soon) | Dataset (coming soon) |

Left: Pipelines for optical flow. A typical pipeline pre-trains models on static datasets,e.g., FlyingChairs, and then evaluates the performance on a target dataset,e.g., Sintel. AutoFlow learns pre-training data which is optimized ona target dataset. Right: Accuracy w.r.t. number of pre-training examples on Sintel.final. Four AutoFlow pre-training examples with augmentation achieve lower errors than 22,872 FlyingChairs pre-training examples with augmentation. The gap between PWC-Net and RAFT becomes small when pre-trained on enough AutoFlow examples.

Abstract

Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications. To automate the process, we present AutoFlow, a simple and effective method to render training data for optical flow that optimizes the performance of a model on a target dataset. AutoFlow takes a layered approach to render synthetic data, where the motion, shape, and appearance of each layer are controlled by learnable hyperparameters. Experimental results show that AutoFlow achieves state-of-the-art accuracy in pre-training both PWC-Net and RAFT.

 

Papers

 

"AutoFlow: Learning a Better Training Set for Optical Flow"
Deqing Sun, Daniel Vlasic, Charles Herrmann, Varun Jampani, Michael Krainin, Huiwen Chang, Ramin Zabih, William T. Freeman, and Ce Liu
Oral presentation, CVPR 2021.
[arXiv][CVF]

Samples

Code

Dataset

Bibtex

@inproceedings{sun2021autoflow,
  title={AutoFlow: Learning a Better Training Set for Optical Flow},
  author={Sun, Deqing and Vlasic, Daniel and Herrmann, Charles and Jampani, Varun and Krainin, Michael
   and Chang, Huiwen and Zabih, Ramin and Freeman, William T and Liu, Ce}, 
    booktitle={CVPR},
  year={2021}
}