Thesis: Vivek Kwatra's PhD Thesis (2005) "Example-based Rendering of Textural Phenomena"

Vivek Kwatra (2005), “Example-based Rendering of Textural Phenomena”PhD Thesis, Georgia Institute of Technology, College of Computing (Advisors: Aaron Bobick, Irfan Essa) [URI], 19-Jul-2005

Abstract

This thesis explores synthesis by example as a paradigm for rendering real-world phenomena. In particular, phenomena that can be visually described as texture are considered. We exploit, for synthesis, the self-repeating nature of the visual elements constituting these texture exemplars. Techniques for unconstrained and constrained/controllable synthesis of both image and video textures are presented. For unconstrained synthesis, we present two robust techniques that can perform spatiotemporal extension, editing, and merging of image and video textures. In one of these techniques, large patches of input texture are automatically aligned and seamlessly stitched with each other to generate realistic-looking images and videos. The second technique is based on iterative optimization of a global energy function that measures the quality of the synthesized texture concerning the given input exemplar. We also present a technique for controllable texture synthesis. In particular, it allows for generating motion-controlled texture animations that follow a specified flow field. Animations synthesized in this fashion maintain structural properties like local shape, size, and orientation of the input texture even as they move according to the specified flow. We cast this problem into an optimization framework that tries to simultaneously satisfy the two (potentially competing) objectives of similarity to the input texture and consistency with the flow field. This optimization is a simple extension of the approach used for unconstrained texture synthesis. A general framework for example-based synthesis and rendering is also presented. This framework provides a design space for constructing example-based rendering algorithms. The goal of such algorithms would be to use texture exemplars to render animations for which certain behavioral characteristics need to be controlled. Our motion-controlled texture synthesis technique is an instantiation of this framework where the characteristic being controlled is motion represented as a flow field.