Realistic Object Acquisition and Display
The core competence of the team is in the acquisition, rendering and display of photo realistic 3D models of real objects and geometry. In order to fully capture the appearance both the 3D geometry as well as the light transport characteristics, i.e., the view and illumination dependent reflection properties, have to be measured precisely. The current focus is on the acquisition outside of a lab environment or the acquisition of multispectral reflectance. In addition, capturing the reflectance of moving, dynamic objects is still to be solved. Besides acquiring reflection properties we also work on reproducing them on novel display devices.
Beyond traditional ways of taking pictures the group is researching novel cameras and algorithms. By modifying the optics, the sensor and by combining multiple different sensors paired with powerful algorithms it is possible to acquire better pictures or pictures showing other modes rather than color. By employing programmable diffractive optics we can shape the PSF of a camera. In such a way, images might be refocused after capture, the contrast might be increased while reducing noise in HDR images. One can capture 3D geometry from image collections and videos, or capture aspects that otherwise are invisible, i.e. temporal, polarization or wavelength effects. By coupling cameras with projectors and compute power the additional information can also be visualized on real world surfaces.
Massively Parallel Programming (GPGPU)
Besides the traditional topic of real-time rendering on graphics cards our research focus on using the massive parallelism in modern GPUs with hundreds of cores for supporting general purpose computing. All aspects of designing algorithms on massively parallel platforms are of interest. For example, we employ GPUs for the simulation of particular visual effects such as diffraction, fluorescence and alike, to perform advanced image processing in real-time, e.g. contrast enhancement or cartoon-style rendering based on edge-avoiding wavelets. Even more important, multiple GPUs are joint to implement large scale numerical solvers, approaching inverse problems, to evaluate Monte Carlo simulations or for enabling large scale 3D reconstruction.
Semantic Image Analysis and Synthesis
Team: Katharina Schwarz
Linking images and natural language opens new application scenarios. For one, providing a natural language interface to the visual world enables localization and reasoning about everyday objects in our environment. Furthermore, by closely analyzing the semantics of objects and by interpreting scenes we work on automatically translating text into visual representations. A key ingredient is harvesting of multiple knowledge sources both in form of texts as well as image and video databases.
Machine Learning for Image and Video Processing
Many image and video processing algorithms have to deal with constantly changing input data as each frame might show different content in a different illumination. We apply machine learning architectures such as deep convolutional neural networks, recurrent networks and alike to enable novel information extraction and enhancement applications. They include image and video deblurring, road lane prediction, appearance robust similarity estimations and measuring aesthetics.
- International Max Planck Research School for Intelligent Systems
- Fraunhofer Institut für Angewandte Optik und Fenmechanik IOF
- DFG - SFB 1233 Robust Vision
- DFG - Emmy Noether (Le 1341/1-1)
- Daimler AG
- NVIDIA Professor Partnership Award