AutoFlow: Learning a Better Training Set for Optical Flow

Deqing Sun Daniel Vlasic Charles Herrmann Varun Jampani Michael Krainin Huiwen Chang
Ramin Zabih William T. Freeman Ce Liu
Google Research
| Paper | Samples | Code (coming soon) | Dataset (available now!) |

Left: Pipelines for optical flow. A typical pipeline pre-trains models on static datasets,e.g., FlyingChairs, and then evaluates the performance on a target dataset,e.g., Sintel. AutoFlow learns pre-training data which is optimized ona target dataset. Right: Accuracy w.r.t. number of pre-training examples on Four AutoFlow pre-training examples with augmentation achieve lower errors than 22,872 FlyingChairs pre-training examples with augmentation. The gap between PWC-Net and RAFT becomes small when pre-trained on enough AutoFlow examples.


Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications. To automate the process, we present AutoFlow, a simple and effective method to render training data for optical flow that optimizes the performance of a model on a target dataset. AutoFlow takes a layered approach to render synthetic data, where the motion, shape, and appearance of each layer are controlled by learnable hyperparameters. Experimental results show that AutoFlow achieves state-of-the-art accuracy in pre-training both PWC-Net and RAFT.




"AutoFlow: Learning a Better Training Set for Optical Flow"
Deqing Sun, Daniel Vlasic, Charles Herrmann, Varun Jampani, Michael Krainin, Huiwen Chang, Ramin Zabih, William T. Freeman, and Ce Liu
Oral presentation, CVPR 2021.





  title={AutoFlow: Learning a Better Training Set for Optical Flow},
  author={Sun, Deqing and Vlasic, Daniel and Herrmann, Charles and Jampani, Varun and Krainin, Michael
   and Chang, Huiwen and Zabih, Ramin and Freeman, William T and Liu, Ce},