Cookies
O website necessita de alguns cookies e outros recursos semelhantes para funcionar. Caso o permita, o INESC TEC irá utilizar cookies para recolher dados sobre as suas visitas, contribuindo, assim, para estatísticas agregadas que permitem melhorar o nosso serviço. Ver mais
Aceitar Rejeitar
  • Menu
Publicações

2025

Human Experts vs. Large Language Models: Evaluating Annotation Scheme and Guidelines Development for Clinical Narratives

Autores
Fernandes, AL; Silvano, P; Guimarães, N; Silva, RR; Munna, TA; Cunha, LF; Leal, A; Campos, R; Jorge, A;

Publicação
Proceedings of Text2Story - Eighth Workshop on Narrative Extraction From Texts held in conjunction with the 47th European Conference on Information Retrieval (ECIR 2025), Lucca, Italy, April 10, 2025.

Abstract
Electronic Health Records (EHRs) contain vast amounts of unstructured narrative text, posing challenges for organization, curation, and automated information extraction in clinical and research settings. Developing effective annotation schemes is crucial for training extraction models, yet it remains complex for both human experts and Large Language Models (LLMs). This study compares human- and LLM-generated annotation schemes and guidelines through an experimental framework. In the first phase, both a human expert and an LLM created annotation schemes based on predefined criteria. In the second phase, experienced annotators applied these schemes following the guidelines. In both cases, the results were qualitatively evaluated using Likert scales. The findings indicate that the human-generated scheme is more comprehensive, coherent, and clear compared to those produced by the LLM. These results align with previous research suggesting that while LLMs show promising performance with respect to text annotation, the same does not apply to the development of annotation schemes, and human validation remains essential to ensure accuracy and reliability. © 2025 Copyright for this paper by its authors.

2025

Strategies and Tools to Support Place-Belongingness in Smart Cities

Autores
Hesam Mohseni; António Correia; Johanna Silvennoinen; Tuomo Kujala; Tommi Kärkkäinen;

Publicação
Computer-Human Interaction Research and Applications

Abstract

2025

Using LLMs to Generate Patient Journeys in Portuguese: an Experiment

Autores
Munna, TA; Fernandes, AL; Silvano, P; Guimarães, N; Jorge, A;

Publicação
Proceedings of Text2Story - Eighth Workshop on Narrative Extraction From Texts held in conjunction with the 47th European Conference on Information Retrieval (ECIR 2025), Lucca, Italy, April 10, 2025.

Abstract
The relationship of a patient with a hospital from admission to discharge is often kept in a series of textual documents that describe the patient’s journey. These documents are important to analyze the different steps of the clinical process and to make aggregated studies of the paths of patients in the hospital. In this paper, we explore the potential of Large Language Models (LLMs) to generate realistic and comprehensive patient journeys in European Portuguese, addressing the scarcity of medical data in this specific context. We employed Google’s Gemini 1.5 Flash model and utilized a dataset of 285 European Portuguese published case reports from the SPMI website, published by the Portuguese Society of Internal Medicine, as references for generating synthetic medical reports. Our methodology involves a sequential approach to generating a synthetic patient journey. Initially, we generate an admission report, followed by a discharge report. Subsequently, we generate a comprehensive patient journey that integrates the admission, multiple daily progress reports, and the discharge into a cohesive narrative. This end-to-end process ensures a realistic and detailed representation of the patient’s clinical pathway as a patient’s journey. The generated reports were rigorously evaluated by medical and linguistic professionals, as well as automatic metrics to measure the inclusion of key medical entities, similarity to the case report, and correct Portuguese variant. Both qualitative and quantitative evaluations confirmed that the generated synthetic reports are predominantly written in European Portuguese without the loss of important medical information from the case reports. This work contributes to developing high-quality synthetic medical data for training LLMs and advancing AI-driven healthcare applications in under-resourced language settings. © 2025 Copyright for this paper by its authors.

2025

Post, Predict, and Rank: Exploring the Relationship Between Social Media Strategy and Higher Education Institution Rankings

Autores
Rocha, B; Figueira, A;

Publicação
INFORMATICS-BASEL

Abstract
In today's competitive higher education sector, institutions increasingly rely on international rankings to secure financial resources, attract top-tier talent, and elevate their global reputation. Simultaneously, these universities have expanded their presence on social media, utilizing sophisticated posting strategies to disseminate information and boost recognition and engagement. This study examines the relationship between higher education institutions' (HEIs') rankings and their social media posting strategies. We gathered and analyzed publications from 18 HEIs featured in a consolidated ranking system, examining various features of their social media posts. To better understand these strategies, we categorized the posts into five predefined topics-engagement, research, image, society, and education. This categorization, combined with Long Short-Term Memory (LSTM) and a Random Forest (RF) algorithm, was utilized to predict social media output in the last five days of each month, achieving successful results. This paper further explores how variations in these social media strategies correlate with the rankings of HEIs. Our findings suggest a nuanced interaction between social media engagement and the perceived prestige of HEIs.

2025

AI and Digital Nomads: Glimpsing the Future Human-Computer Interaction

Autores
Marcos Antonio de Almeida; António Correia; Carlos Eduardo Barbosa; Jano Moreira de Souza; Daniel Schneider;

Publicação
Computer-Human Interaction Research and Applications

Abstract

2025

Can ChatGPT Suggest Patterns? An Exploratory Study About Answers Given by AI-Assisted Tools to Design Problems

Autores
Maranhao, JJ Jr; Correia, FF; Guerra, EM;

Publicação
AGILE PROCESSES IN SOFTWARE ENGINEERING AND EXTREME PROGRAMMING-WORKSHOPS, XP 2024 WORKSHOPS

Abstract
General-purpose AI-assisted tools, such as ChatGPT, have recently gained much attention from the media and the general public. That raised questions about in which tasks we can apply such a tool. A good code design is essential for agile software development to keep it ready for change. In this context, identifying which design pattern can be appropriate for a given scenario can be considered an advanced skill that requires a high degree of abstraction and a good knowledge of object orientation. This paper aims to perform an exploratory study investigating the effectiveness of an AI-assisted tool in assisting developers in choosing a design pattern to solve design scenarios. To reach this goal, we gathered 56 existing questions used by teachers and public tenders that provide a concrete context and ask which design pattern would be suitable. We submitted these questions to ChatGPT and analyzed the answers. We found that 93% of the questions were answered correctly with a good level of detail, demonstrating the potential of such a tool as a valuable resource to help developers to apply design patterns and make design decisions.

  • 13
  • 4212