Content-aware Optimization for Image and Video Applications

Abstract

In recent years the (stereoscopic) image/video retargeting fast rises to become an important technology due to many display devices with different resolutions such as TVs smart phones tablet PCs 3D displays and so on Most non-linear warping methods used in image/video resizing are to preserve the aspect ratios of prominent regions and distort unimportant contents However these methods sometimes cause significant distortion of objects or structure lines depth distortion and unnatural temporal motions because of ignoring the consistency in object deformation Therefore a floating boundary with volumetric warping and object-aware cropping is proposed to address those problems In the proposed scheme visually salient objects in the space-time domain are deformed as rigidly and as consistently as possible by using information from matched objects and content-aware boundary constraints The content-aware boundary constraints can retain visually salient contents in a fixed region with a desired resolution and aspect ratio called critical region during warping Volumetric cropping with the fixed critical region is then performed to adjust stereoscopic videos to the desired aspect ratios The strategies of warping and cropping using floating boundaries and spatiotemporal constraints enable our method to consistently preserve the temporal motions and spatial shapes of visually salient volumetric objects in the left and right videos as much as possible thus leading to good content-aware retargeting In addition by considering shape motion and disparity preservation the proposed scheme can be applied to various media including images stereoscopic images videos and stereoscopic videos Qualitative and quantitative analyses on stereoscopic videos with diverse camera and considerable motions demonstrate a clear superiority of the proposed method over related methods in terms of retargeting quality In addition to the resizing method we extend our warping technique to maps Tourist and destination maps are thematic maps designed to represent specific themes in maps The road network topologies in these maps are generally more important than the geometric accuracy of roads A road network warping method is proposed to facilitate map generation and improve theme representation in maps The basic idea is deforming a road network to meet a user-specified mental map while an optimization process is performed to propagate distortions originating from road network warping To generate a map the proposed method includes algorithms for estimating road significance and for deforming a road network according to various geometric and aesthetic constraints The proposed method can produce an iconic mark of a theme from a road network and meet a user-specified mental map Therefore the resulting map can serve as a tourist or destination map that not only provides visual aids for route planning and navigation tasks but also visually emphasizes the presentation of a theme in a map for the purpose of advertising In the experiments the demonstrations of map generations show that our method enables map generation systems to generate deformed tourist and destination maps efficiently QR code is generally used for embedding messages such that people can conveniently use mobile devices to capture the QR code and acquire information through a QR code reader In the past the design of QR code generators only aimed to achieve high decodability and the produced QR codes usually look like random black-and-white patterns without visual semantics In recent years researchers have been tried to endow the QR code with aesthetic elements and QR code beautification has been formulated as an optimization problem that minimizes the visual perception distortion subject to acceptable decoding rate However the visual quality of the QR code generated by existing methods still leaves much to be desired In this work we propose a two-stage approach to generate QR code with high quality visual content In the first stage a baseline QR code with reliable decodability but poor visual quality is first synthesized based on the Gauss-Jordan elimination procedure In the second stage a rendering mechanism is designed to improve the visual quality while avoid affecting the decodability of the QR code The experimental results show that the proposed method substantially enhances the appearance of the QR code and the processing complexity is near real-time

Date of Award	2015 Jun 16
Original language	English
Supervisor	Tong-Yee Lee (Supervisor)