INESC TEC develops immersive tools and contents for education and vocational training

INESC TEC develops immersive tools and contents for education and vocational training

The Centre for Telecommunications and Multimedia (CTM) and the Centre for Information Systems and Computer Graphics (CSIG) at INESC TEC have recently concluded project CNG (New Generation Contents), whose main result was to develop a prototype for an interactive game that could be used by both teachers and students to help them teach and learn mathematics.

26th September 2014

With a duration of three years, the purpose of this project was to use immersive environments as an innovative way of providing appealing contents, while allowing users to benefit from a 3D audio and video experience. This new way of experiencing contents will give users suitable spatial perceptions in various areas of knowledge, especially mathematics, which are truly innovative and geared towards younger audiences.

The researchers involved in this project developed a new Mathematics teaching and learning tool. The prototype developed consists of an immersive and interactive game which uses a Kinect device to capture the natural gestures of users. The game was specifically designed for pre-school and primary school students to help them learn mathematics in a fun way, using two components – SinglePlayer and Multiplayer.

The game’s Multiplayer component includes a system to create and manage game rooms where the players can meet and play. This component benefits from new generation networks because it allows players to exchange videos. Simultaneously, the project investigated efficient approaches to transmit 3D video contents in IP (“Internet Protocol”) networks, providing a personalised and low-cost audio-visual and immersive experience to the user through IP networks.

In that sense, a prototype of a client-server system was developed that, contrarily to the existing systems based on video multiview coding technology that require large bandwidths and/or high processing capacity in the receiver, maximise the efficiency of the resources available, namely the network and the terminal resources, presenting multiview contents to different audiences in an innovative and personalised way.

Practically speaking, the system can adapt, in a dynamic way, the perspective of the 3D visual scene that is being presented to the user at each instant, in accordance with the focus of attention of the user in the scene. Simultaneously, the rate of the data flow that is being transmitted (and consequently the quality, although not necessarily in a direct/linear way) is adapted according to the available bandwidth. To develop this prototype, the researchers involved used open source technology and open standards.

The INESC TEC team involved in this project included Augusto Sousa, António Coelho, Alexandre Carvalho, Maximino Bessa and Luís Magalhães from CSIG, and Teresa Andrade from CTM.

Other than INESC TEC, other partners involved in the project include iZone Interactive Media, iZone Knowledge Systems, Microsoft Portugal, Diferente Jogo, the University of Aveiro, the Faculty of Engineering of the University of Porto (FEUP) and the Center of Computer Graphics of the University of Minho.

The INESC TEC researchers mentioned in this article are associated with the following partner institutions: INESC Porto, FEUP and UTAD.