Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by CTM

2023

In-Context Impersonation Reveals Large Language Models' Strengths and Biases

Authors
Salewski, L; Alaniz, S; Rio-Torto, I; Schulz, E; Akata, Z;

Publication
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023)

Abstract
In everyday conversations, humans can take on different roles and adapt their vocabulary to their chosen roles. We explore whether LLMs can take on, that is impersonate, different roles when they generate text in-context. We ask LLMs to assume different personas before solving vision and language tasks. We do this by prefixing the prompt with a persona that is associated either with a social identity or domain expertise. In a multi-armed bandit task, we find that LLMs pretending to be children of different ages recover human-like developmental stages of exploration. In a language-based reasoning task, we find that LLMs impersonating domain experts perform better than LLMs impersonating non-domain experts. Finally, we test whether LLMs' impersonations are complementary to visual information when describing different categories. We find that impersonation can improve performance: an LLM prompted to be a bird expert describes birds better than one prompted to be a car expert. However, impersonation can also uncover LLMs' biases: an LLM prompted to be a man describes cars better than one prompted to be a woman. These findings demonstrate that LLMs are capable of taking on diverse roles and that this in-context impersonation can be used to uncover their strengths and hidden biases. Our code is available at https://github.com/ExplainableML/in-context-impersonation.

2023

Towards an airtightness compliance tool based on machine learning models for naturally ventilated dwellings

Authors
Cardoso, VEM; Simoes, ML; Ramos, NMM; Almeida, RMSF; Almeida, M; Sanhudo, L; Fernandes, JND;

Publication
ENERGY AND BUILDINGS

Abstract
Physical models and probabilistic applications often guide the study and characterization of natural phenomena in engineering. Such is the case of the study of air change rates (ACHs) in buildings for their complex mechanisms and high variability. It is not uncommon for the referred applications to be costly and impractical in both time and computation, resulting in the use of simplified methodologies and setups. The incorporation of airtightness limits to quantify adequate ACHs in national transpositions of the Energy Performance Building Directive (EPBD) exemplifies the issue. This research presents a roadmap for developing an alternative instrument, a compliance tool built with a Machine Learning (ML) framework, that overcomes some simplification issues regarding policy implementation while fulfilling practitioners' needs and general societal use. It relies on dwellings' terrain, geometric and airtightness characteristics, and meteorological data. Results from previous work on a region with a mild heating season in southern Europe apply in training and testing the proposed tool. The tool outputs numerical information on the air change rates performance of the building envelope, and a label, accordingly. On the test set, the best regressor showed mean absolute errors (MAE) below 1.02% for all the response variables, while the best classifier presented an average accuracy of 97.32%. These results are promising for the generalization of this methodology, with potential for application at regional, national, and European Union levels. The developed tool could be a complementary asset to energy certification programmes of either public or private initiatives. (c) 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

2023

Brain activation by a VR-based motor imagery and observation task: An fMRI study

Authors
Nunes, D; Vourvopoulos, A; Blanco Mora, DA; Jorge, C; Fernandes, J; Bermudez I Badia, S; Figueiredo, P;

Publication
PloS one

Abstract
Training motor imagery (MI) and motor observation (MO) tasks is being intensively exploited to promote brain plasticity in the context of post-stroke rehabilitation strategies. This may benefit from the use of closed-loop neurofeedback, embedded in brain-computer interfaces (BCI's) to provide an alternative non-muscular channel, which may be further augmented through embodied feedback delivered through virtual reality (VR). Here, we used functional magnetic resonance imaging (fMRI) in a group of healthy adults to map brain activation elicited by an ecologically-valid task based on a VR-BCI paradigm called NeuRow, whereby participants perform MI of rowing with the left or right arm (i.e., MI), while observing the corresponding movement of the virtual arm of an avatar (i.e., MO), on the same side, in a first-person perspective. We found that this MI-MO task elicited stronger brain activation when compared with a conventional MI-only task based on the Graz BCI paradigm, as well as to an overt motor execution task. It recruited large portions of the parietal and occipital cortices in addition to the somatomotor and premotor cortices, including the mirror neuron system (MNS), associated with action observation, as well as visual areas related with visual attention and motion processing. Overall, our findings suggest that the virtual representation of the arms in an ecologically-valid MI-MO task engage the brain beyond conventional MI tasks, which we propose could be explored for more effective neurorehabilitation protocols. Copyright: © 2023 Nunes et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

2023

Zero-shot face recognition: Improving the discriminability of visual face features using a Semantic-Guided Attention Model

Authors
Patricio, C; Neves, JC;

Publication
EXPERT SYSTEMS WITH APPLICATIONS

Abstract
Zero-shot learning enables the recognition of classes not seen during training through the use of semantic information comprising a visual description of the class either in textual or attribute form. Despite the advances in the performance of zero-shot learning methods, most of the works do not explicitly exploit the correlation between the visual attributes of the image and their corresponding semantic attributes for learning discriminative visual features. In this paper, we introduce an attention-based strategy for deriving features from the image regions regarding the most prominent attributes of the image class. In particular, we train a Convolutional Neural Network (CNN) for image attribute prediction and use a gradient-weighted method for deriving the attention activation maps of the most salient image attributes. These maps are then incorporated into the feature extraction process of Zero-Shot Learning (ZSL) approaches for improving the discriminability of the features produced through the implicit inclusion of semantic information. For experimental validation, the performance of state-of-the-art ZSL methods was determined using features with and without the proposed attention model. Surprisingly, we discover that the proposed strategy degrades the performance of ZSL methods in classical ZSL datasets (AWA2), but it can significantly improve performance when using face datasets. Our experiments show that these results are a consequence of the interpretability of the dataset attributes, suggesting that existing ZSL datasets attributes are, in most cases, difficult to be identifiable in the image. Source code is available at https://github.com/CristianoPatricio/SGAM.

2023

Scaling VR Video Conferencing

Authors
Dasari, M; Lu, E; Farb, MW; Pereira, N; Liang, I; Rowe, A;

Publication
2023 IEEE CONFERENCE VIRTUAL REALITY AND 3D USER INTERFACES, VR

Abstract
Virtual Reality (VR) telepresence platforms are being challenged to support live performances, sporting events, and conferences with thousands of users across seamless virtual worlds. Current systems have struggled to meet these demands which has led to high-profile performance events with groups of users isolated in parallel sessions. The core difference in scaling VR environments compared to classic 2D video content delivery comes from the dynamic peer-to-peer spatial dependence on communication. Users have many pair-wise interactions that grow and shrink as they explore spaces. In this paper, we discuss the challenges of VR scaling and present an architecture that supports hundreds of users with spatial audio and video in a single virtual environment. We leverage the property of spatial locality with two key optimizations: (1) a Quality of Service (QoS) scheme to prioritize audio and video traffic based on users' locality, and (2) a resource manager that allocates client connections across multiple servers based on user proximity within the virtual world. Through real-world deployments and extensive evaluations under real and simulated environments, we demonstrate the scalability of our platform while showing improved QoS compared with existing approaches.

2023

Invasive and Minimally Invasive Evaluation of Diffusion Properties of Sugar in Muscle

Authors
Pinheiro, MR; Tuchin, VV; Oliveira, LM;

Publication
IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS

Abstract
In this article, the use of diffuse reflectance (R-d) spectroscopy is explored to evaluate the diffusion properties of water and sucrose in skeletal muscle during optical clearing treatments. Treating muscle samples with sucrose-water solutions with different osmolarities, collimated transmittance (T-c) and R-d measurements were performed to obtain the diffusion time (t) and the diffusion coefficient (D) values that characterize the unique water and sucrose fluxes in the muscle and also the optical clearing mechanisms designated as tissue dehydration and refractive index matching. Considering the R-d measurements, the estimated t and D values for water in the muscle were 63.1s and 1.72x10(-6) cm(2)/s, while the ones estimated for sucrose were 261s and 4.86x10(-7) cm(2)/s. Comparing these values with the ones estimated from the T-c measurements, the relative differences observed for t and D were 1.6% and 2.8% in the case of water and 0.3% and 0.4% in the case of sucrose.

  • 46
  • 317