1. Difference Map Generation
Difference Map is to help us locate the flaws in the initial estimated intermediate flow.
The challenge posed by large motion plays a crucial role in the task of Video Frame Interpolation (VFI). Existing methods are often constrained by limited receptive fields, resulting in suboptimal performance when dealing with scenarios with large motion. In this paper, we introduce a novel pipeline, which effectively integrating global-level information, to alleviate dilemmas associated with large motion.
Specifically, we first estimate a pair of initial intermediate flows using high-resolution feature map for extracting local details. Then, we incorporate a sparse global matching branch to compensate for flow estimation, which consists of identifying flaws in initial flows and generating sparse flow compensation with a global receptive field. Finally, we adaptively merge the initial flow estimation with global flow compensation, yielding a more accurate intermediate flow.
To evaluate the model's effectiveness in handling large motion, we carefully curated the most challenging subset from commonly used benchmarks. As a result, our method demonstrates state-of-the-art performance on these demanding large motion subsets.
Difference Map is to help us locate the flaws in the initial estimated intermediate flow.
We use sparse global matching to introduce global information into our estimated intermediate flows.
The flow needs to be shifted to the correct coordinate.
The output of sparse global matching may be incorrect, so we merge it with the initial estimated flow to create a more accurate flow.