Fraunhofer HHI Put First European Volumetric Video Studio Into Operation

Fraunhofer HHI has put into operation a volumetric video studio in October 2017 and since then, three professional commissioned productions have already been carried out in this studio and two more are planned until the end of February.

The studio, located at Fraunhofer HHI, serves primarily as an experimental laboratory with the purpose to further develop the
basic technology of 3D Human Body Reconstruction, an innovative procedure that enables a realistic, authentic and particularly natural 3D image of moving people in virtual worlds. The commercial use of the developed technology is being taken over, in the short term, by VoluCap GmbH (Volumetric Capture Studio Babelsberg GmbH) in Potsdam-Babelsberg, which was founded specifically for this purpose.

The necessity to relocate the commercial production to a company is due in particular to the rapid market development of
virtual and augmented reality and the high demand for volumetric video content resulting from this. The company formation enables the Fraunhofer HHI, as an institute for scientific research, to focus more on the technical and scientific advancement of the HBR basic technology.

Last November, the first professional commissioned production was carried out in the studio of Fraunhofer HHI and is part of the VR experience “Ein ganzes Leben” (A Whole Life), which premiered at the Deutsche Kinemathek, the film and television museum in Berlin, with the special exhibition “Die Ufa – Geschichte einer Marke” (Ufa – History of a Brand). Additional productions from the volumetric video studio at Fraunhofer HHI will be presented to the public in several talks and presentations at this year’s Berlinale.

The 3D Human Body Reconstruction technology captures real people with multiple cameras at the same time and creates naturally moving dynamic 3D models, which can be observed from arbitrary viewpoints in the virtual scene. The HBR capture system consists of 16 stereo cameras that create 3D information from all different viewpoints around the person, similar to how humans perceive with their two eyes. This information is then transformed into a consistent, natural and dynamic 3D representation of the person.

Post-processing modules adapt the data structures and formats and allow direct integration in standardized post-production workflows for VR and AR applications. In contrast to classical animation of virtual characters, facial expressions as well as moving clothes are captured from video information and reconstructed at high geometrical detail and texture quality. This means that the maximum of naturalness of the visual impression is achieved and the artificiality of avatars can be avoided. Due to the standardized output format, further post-processing of the dynamic 3D model is possible.

Beyond dynamic 3D models processing, an integrated multi-camera and lighting system for full 360-degree acquisition of persons has also been developed. The system enables a controlled, freely configurable and diffuse illumination of the persons, which allows both a subsequent synthetic illumination of the dynamic 3D-models in VR worlds (re-lighting) as well as a targeted dynamic scenic illumination with special lighting moods (campfire, blue light, etc.).