IPAB Seminar - 07/11/2019

 

Neill Campbell 

 

TITLE:  Nonparametric Models and Uncertainty Propagation in Generative Deep Networks

 

ABSRACT: Generative models offer a host of advantages to intelligent systems; for example, the ability to imagine and synthesise new data and futures, produce simplified representations for reasoning, operate with partial and missing data, and coalesce information from different modalities. Unfortunately, unsupervised learning of generative models has always been a challenging topic in vision due to the huge degrees of freedom involved and the difficulties in defining appropriate likelihood functions over images. Recently, however, advances in the field have been driven by the application of deep networks to the learning of likelihood functions. The literature is roughly divided into two approaches: methods that focus on spanning the true data distribution (Variational Auto-Encoders - VAE) or those that ensure high quality samples (Generative Adversarial Networks - GAN). In some respects, these works trade-off against one another and we would argue that both have sacrificed some of the advantageous theoretical properties from previous probabilistic approaches, in particular the treatment of uncertainty throughout the model. We present an argument in favour of the use of non-parametric models and uncertainty propagation to provide additional advantages to deep generative models. In particular: the provision of priors that encourage sharing in the latent space; attempting to capture true epistemic uncertainty in the latent space (via direct access to the posterior); the propagation of uncertainties throughout the generative process; and accounting for the aleatoric uncertainty in the model output. We first look at the aleatoric uncertainty by modelling the output distribution of a VAE directly. Output quality from a VAE is often limited by the loss functions (e.g. L2 or spherical/diagonal Gaussian likelihood). In contrast, we estimate a structured precision matrix that is able to capture correlations in the residual errors of a standard VAE and show that sampling from this output distribution produces images that match the statistics of the original training data. We demonstrate the efficacy of this approach on a denosing task that shows it correctly models the high-frequency content of the images and has better captured the statistics of the dataset. We then consider capturing and propagating epistemic uncertainty from the latent space to the output. We combine a non-parametric latent variable model, encoding a smooth and probabilistic latent space, with a stochastic network that can learn non-Gaussian likelihoods and maintains uncertainty. The experiments challenge the current trend in unsupervised learning towards maximum likelihood training of increasingly large parametric models with increasingly large datasets. Early results suggest that, by propagating uncertainty throughout the model, we can improve upon the performance of two standard generative deep learning models, a VAE and an InfoGAN, with comparable architectures and achieve similar results with far smaller training datasets (data efficiency). This involves collaborative work with Alessandro Di Martino and Garoe Dorta Perez at the University of Bath; Carl Henrik Ek and Erik Bodin at the University of Bristol; Lourdes Agapito at UCL; and Ivor Simpson and Sara Vicente at Anthropics Ltd. Bio: Neill is a Royal Society Industry Fellow and a Senior Lecturer (Associate Professor) in Computer Vision, Graphics and Machine Learning at the University of Bath. He also holds an Honorary Associate Professor position in the Vision and Graphics Group in the Department of Computer Science at University College London where he was formerly a Research Associate working with Jan Kautz and Simon Prince on synthesizing and editing photorealistic visual objects. He completed his PhD in the Machine Intelligence Laboratory at the Department of Engineering at the University of Cambridge under the supervision of Roberto Cipolla. His main area of research involves learning models of shape (2D and 3D), motion and appearance from images. As part of his Fellowship he works with the Foundry (a world leading supplier of software tools to the creative industries) on the development of “intelligent software tools” where machine learning is used to make creative visual computing tasks easier for users.

 

Tobias Ritschel

 

TITLE: Deep Appearance capture and reproduction

 

 ABSTRACT: I will discuss different learning-based methods to acquire and render  visual appearance capture. Starting by replacing individual component in the graphics pipeline, we will see how reproduction and capture can  interplay. Further, I will suggest approaches in which the classic  rendering pipeline is changed and individual steps start to blend.  Ultimately I will discuss how close we are to optimizing over the entire  graphics pipeline.  BIO (long):  Tobias has received his PhD from Saarland University (MPI) in 2009. He  was a post-doctoral researcher and Telecom ParisTech / CNRS 2009-10 and  a Senior Researcher at MPI 2010-15. Tobias was appointed Senior Lecturer  at University College London in 2015 and named Professor of Computer  Graphics in 2019. His work has received the EG Dissertation (2010) and  Young Researcher Award (2014). He frequently serves on the IPCs of  SIGGRAPH, Eurographics, EGSR as well as CVPR, ICCV and ECCV. He also is  an Associated Editor of Springer Visual Computing and IEEE, Trans.  Visualization and Computer Graphics and served as general chair for the  EG Symposium on Geometry Processing in 2015 and the EG Symposium of  Rendering in 2020. His interests include Image Synthesis, in particular  its efficiency (Global Illumination, Image-based rendering, VR) and  computational models of Human Visual Perception (Stereo, Color and  Luminance Contrast). His work now frequently makes use of applied AI, in  particular less- and unsupervised learning, learning in advanced domains  (3D point clouds/meshes/volumes) as well as general back-propagatable  graphics architectures.  

Nov 07 2019 -

IPAB Seminar - 07/11/2019

Neill Campbell (University of Bath), Tobias Ritschel (University College London)

IF, G.03