Learning 3D Human and Object Pose
Speaker: Kostas Daniilidis, GRASP Laboratory, University of Pennsylvania

Abstract: Computer vision has been very successful in extracting 3D shape and
pose from multiple views or from depth sensors. Today, though, we have
an abundance of 2D images as well as an increasing amount of 3D
exemplars. In this talk, I will present results on 3D reconstruction
for given class of shapes: humans or object categories. 3D shape is
encoded with keypoints and new instances of a class can be encoded as
sparse representations from a shape dictionary if we correctly detect
these keypoints in images. Such keypoints can be detected and
localized with Convolutional Neural Networks whose output are
probabilistic distributions of 2D keypoints. By considering them as
latent variables we can extract 3D shape and pose using
Expectation-Maximization. We ask then the question: whether such a
two-step approach could be improved by an end-to-end learning of the
3D shape from images. Indeed, it turns out that a coarse-to-fine
approach in depth beats the two-step estimation and advances the
state of the art. We show applications of class-based reconstruction
using airborne images of humans in motion. This is joint work with
Xiaowei Zhou, George Pavlakos, Kosta Derpanis, Menglong Zhu, and
Spyros Leonardos.

Bio:Kostas Daniilidis is the Ruth Yalom Stone Professor of Computer and
Information Science at the University of Pennsylvania where he has
been faculty since 1998. He is an IEEE Fellow.
He was the director of the interdisciplinary GRASP laboratory from
2008 to 2013, Associate Dean for Graduate Education from 2012-2016,
and Director of Online Learning since 2016.
He obtained his undergraduate degree in Electrical Engineering from
the National Technical University of Athens, 1986, and his PhD (Dr.rer.nat.) in Computer
Science from the University of Karlsruhe, 1992, under the supervision
of Hans-Hellmut Nagel. He was Associate Editor of IEEE Transactions
on Pattern Analysis and Machine Intelligence from 2003 to 2007. He
co-chaired with Pollefeys IEEE 3DPVT 2006, and he was Program co-chair
of ECCV 2010. His most cited works have been on visual odometry,
omnidirectional vision, 3D pose estimation, 3D registration, hand-eye
calibration, structure from motion, and image matching. Kostas’ main
interest today is in deep learning of 3D representations, data
association, event-based cameras, semantic localization and mapping,
and vision based manipulation.