IPAB Seminar - 08/05/2018

Talk Title:  Monocular 3D Pose, Shape and Appearance Modeling for Multiple People in Natural Scenes. Towards Interaction and Fine Grained Human Behavior Understanding

Abstract: Human sensing has greatly benefited from recent advances in deep learning, parametric human modeling, and the availability of large scale 2d and 3d datasets. However, existing models make strong assumptions about the scene, considering either a single person per image, full views of the person, a simple background or many cameras. In this talk, Professor Sminchisescu  will describe approaches that combine deep multi-task neural networks and parametric human and scene models, towards an automatic monocular visual sensing system for interacting people, which infers the 2d and 3d pose and shape of multiple people from a single image, automatically integrates scene constraints, and extends to video by optimally solving the temporal assignment and pose smoothness problem while preserving image alignment fidelity. 

Besides human sensing, the framework supports several modeling capabilities including realistic human appearance transfer as well as human behavior understanding. In this context Professor Sminchisescu  will introduce a new, fine-grained action classification and localization task defined on non-staged videos, recorded during robot-assisted therapy sessions of children with autism, and aimed at estimating their valence-arousal, automating therapy and quantitatively measuring progress.  

Time permitting, he may briefly review ongoing work on deep reinforcement learning for visual recognition as well as matrix backpropagation methodology that allows the training of deep structured models with layers implementing global operations like SVD, eigen-decomposition, projectors, or relaxations, and supporting, the end-to-end refinement of deep spectral clustering, higher-order pooling or graph-matching models.

This is joint work E. Marinoiu, A. Zanfir, M. Zanfir, A. Popa, A. Pirinen, C. Ionescu, and D. Nilsson.

Bio: Cristian Sminchisescu is a Professor in the Department of Mathematics, Faculty of Engineering at Lund University and a Research Scientist at Google Research in Zurich. He has a doctorate in computer science and applied mathematics from INRIA and has done postdoctoral research in the AI Lab at the University of Toronto. Prior to Lund he was a faculty in Chicago and Bonn. He has been an associate editor for IEEE PAMI since 2010, an area chair for CVPR, ECCV, and ICCV, and a program chair for ECCV 2018 in Munich. His work has been funded by the NSF, the Swedish, Romanian, German Science Foundations, the European Commission under a Marie Curie Excellence Grant, and the ERC under a Consolidator Grant.