Video Reordering with Optical Flows and Autoencoder

Abstract

To solve the general video resequencing problem we propose a novel deep learning framework to generate the natural result videos with smooth motion Given an unordered image collection or a video we first extract the latent vectors from the images/video frames by a novel architecture we propose Then we build a complete graph with the distance between latent vectors Three different path finding algorithms are used to traverse the graph for producing video sequence results which correspond to three applications of our framework: original video reconstruction in-between frames insertion and video resequencing To ensure the motion of the resulting videos is “as smooth and reasonable as possible” we use optical flows as the constraints in the path finding algorithms and the network architecture we proposed is used to compute the difference of the optical flows The experimental evaluation demonstrates that our proposed network has better performance than the previous work on the feature extraction and the appealing result videos also show that our framework can be applied on many styles of videos or unordered image collection including cartoon and realistic videos without unappealing motion problems in previous study

Date of Award	2020
Original language	English
Supervisor	Tong-Yee Lee (Supervisor)