INESC TEC technology allows users to watch 3D concerts and choose most interesting perspective

INESC TEC technology allows users to watch 3D concerts and choose most interesting perspective

The “Personalized Multiview Environment”, developed at INESC TEC’s Centre for Telecommunications and Multimedia (CTM) together with FEUP as part of the research line “Media, Arts & Technologies (MAT)”, enables immersive personalised experiences, presenting 3D contents to the users based on their preferences.

18th June 2015

The main goal with this solution is to increase the quality of the experience of the users who are consuming audiovisual content, giving them the sensation that they are really “in loco”.

The “Personalized Multiview Environment” uses video capture and multiview coding, with multiple 3D cameras positioned in an arc throughout the entire scenario, in order to capture multiple perspectives from the scene.

“The prototype will provide a realistic experience to the user in a transparent way, according to their interests, with the best quality possible with cost-effective solutions,” explains Maria Teresa Andrade, the researcher in charge of this project, together with Tiago Costa.

The current technology “Personalized Multiview Environment” uses a webcam in the receiver, which films the viewer, and a processing module that follows the head movements, and understands the areas that the used focused on at each instant, sending that information to the server.

“It is as if you are really at the concert , and you are able to turn to your left and focus on specific musicians,” the INESC TEC researcher states.

In the future, the goal is to use an inference model based on probabilistic techniques, with which the server will be able to predict the users’ head movements (based on past records), sending to the client, in a timely manner, a perspective of the scene most likely to be desired by the viewer.

This project – "NORTE-07-0124-FEDER-000061" – is funded by the North Portugal Regional Operational Programme ("ON.2 - O Novo Norte"), the National Strategic Reference Framework (NSRF), through the European Regional Development Fund (ERDF), and by National Funds through the Foundation for Science and Technology (FCT). This prototype was also developed with funds from the project QREN-CNG (Next Generation Contents), where FEUP is also partner.

The INESC TEC researchers mentioned in this article are associated with the following partner institutions: UP-FEUP and INESC TEC.