Inverse Rendering For Computer Graphics

Author : Fujun Luan
Publisher :
ISBN 13 :
Total Pages : 0 pages
Book Rating : 4.:/5 (14 download)

DOWNLOAD NOW!

Download or read book Forward and Inverse Rendering with Gradient-based Optimization written by Fujun Luan and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Photorealistic rendering is a fundamental component of computer graphics. To render photorealistic 2D images that faithfully represent our 3D world, efforts have been devoted into developing compact scientific models and efficient algorithms that are capable of simulating lighting and physics to capture appearance using the geometry, reflectance and illumination. Inverse Rendering, on the other hand, focuses on recovering unknown scene attributes from photograph measurements by optimization, typically through analysis-by-synthesis technique, or more recently, differentiable rendering. In this thesis, we propose solutions to a few such challenging forward and inverse graphics problems, as described below. The first work is a forward rendering (SIGGRAPH 2020) approach to speeding up Markov chain Monte Carlo (MCMC) rendering with derivatives from recent rendering engines that support end-to-end differentiation. In this work, we build upon Langevin dynamics and propose a suite of Langevin Monte Carlo algorithms with gradient-based adaptation for efficient photorealistic rendering of scenes with complex light transport effects, such as caustics, interreflections, and occlusions. In inverse rendering, we develop multiple solutions for recovering material and shape. In our second work (ICCP 2020) we focus on learning-based inverse subsurface scattering. Given images of transcluent objects, of unknown shape and lighting, we aim to use learning to infer the optical parameters controlling subsurface scattering of light inside the objects. With physics-based priors in the learning framework, we obtain strong improvements in both parameter estimation accuracy and appearance reproduction compared to traditional networks. Our third work is a unified shape and appearance reconstruction framework (EGSR 2021) using differentiable rendering for 3D object scanning. We tackle this problem by introducing a new analysis-by-synthesis technique capable of producing high-quality reconstructions through robust coarse-to-fine optimization and physics-based differentiable rendering. We demonstrate the effectiveness of our method on real-world objects captured with handheld cameras that outperforms previous state-of-the-art approaches. In Appendix A, we tackle the challenge of automatically generating procedural representation of fiber-based yarn models for cloth rendering (SIGGRAPH 2016, EGSR 2017). Appendix B has slightly deviated from rendering and focuses on image style transfer techniques (CVPR 2017, EGSR 2018). With recent advances in physics-based differentiable rendering, we have taken first steps to speeding up MCMC forward rendering with first-order gradients, improving learning-based inverse subsurface scattering, and introducing an end-to-end differentiable rendering pipeline for high-quality handheld object scanning. As for next steps, a promising future direction for computer graphics would be learning-based neural rendering methods that aim for AR/VR-based applications. Exploring differentiable rendering for improving traditional 3D reconstruction would be another promising topic, thanks to the theoretical and practical breakthroughs in the corresponding field.