Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

2026

Comparing LLM and expert assessments of journal quality

Authors
Mirka Saarela; Janne Pölönen; Anna-Kaarina Linna; Leena Wahlfors; António Correia; Tommi Kärkkäinen;

Publication
Scientometrics

Abstract
Abstract Some performance-based research funding systems rely on expert-assigned journal rankings to allocate resources and guide research evaluation. In Finland, the JuFo system provides journal rankings, determined by experts who assess journals using available metadata, such as bibliometric indicators, alongside qualitative judgment. While prior work has explored machine learning approaches to approximate these rankings, the recent emergence of large language models (LLMs) offers new possibilities for automated, data-driven evaluation. In this study, we examine how well LLMs can replicate JuFo rankings when given the same structured information available to experts, including citation metrics, disciplinary assignments, and publisher metadata. We systematically compare LLM predictions to expert-assigned JuFo ranks using a confusion-matrix analysis to identify cases of alignment and deviation. Our research addresses two key questions: (1) how accurately LLMs estimate journal rankings, and (2) in which situations their predictions diverge from expert judgments and which factors explain these discrepancies. Our findings show that LLMs approximate expert-assigned rankings with high overall accuracy, with most errors occurring between adjacent levels. However, their performance varies systematically across disciplines, and they tend to under-predict top-tier journals, particularly in social sciences and humanities fields.

2026

Crisis or Redemption with AI and Robotics? The Dawn of a New Era

Authors
Silva, MF; Tokhi, MO; Ferreira, MIA; Malheiro, B; Guedes, P; Ferreira, P; Costa, MT;

Publication
Lecture Notes in Networks and Systems

Abstract

2026

Ethical Considerations in the Context of AI-Driven Misinformation Detection

Authors
Ettore Barbagallo; Guillaume Gadek; Géraud Faye; Nina Khairova; Chirag Arora; Dilhan Thilakarathne; Karen Joisten; Sónia Teixeira; Juan M. Durán; Manuel Barrantes;

Publication
Handbook of Human-AI Collaboration

Abstract
Abstract Misinformation poses one of the most urgent challenges of our society and raises the question of how to deal with it and manage its rapid spread. To address this problem, a promising approach relies on AI-based misinformation detection. This chapter of the book offers a critical analysis of the ethical implications associated with the design, deployment, and use of misinformation detectors (MDs). Designing and deploying an MD—an AI system that automatically identifies misinformation—is a complex undertaking that requires an interdisciplinary approach, as the challenges faced by MD designers and deployers encompass not only technical aspects, but also linguistic, sociological, political, and especially ethical dimensions. Our analysis is ethics-oriented and follows two main lines of inquiry: (1) Ethics by Design, which focuses on issues related to the design process of an MD, and (2) Ethics of Impact, which addresses the intended and unintended effects of MD deployment and use.

2026

From the Margin to the Centre: Ethnomethodology as a Tool for Situating Cultural Insensitivities in AI Through the Lens of Music-Making

Authors
António Correia; Hesam Mohseni; Pieta-Anniina Sikström; Tommi Kärkkäinen;

Publication
2026 International Conference on Artificial Intelligence, Computer, Data Sciences and Applications (ACDSA)

Abstract

2026

Mapping Ethics in EPS@ISEP Robotics Projects

Authors
Malheiro, BA; Guedes, P; Silva, MF; Ferreira, P;

Publication
CRISIS OR REDEMPTION WITH AI AND ROBOTICS? THE DAWN OF A NEW ERA, ICRES 2025

Abstract
The European Project Semester (EPS), offered by the Instituto Superior de Engenharia do Porto (ISEP), is a capstone programme designed for undergraduate students in engineering, product design, and business. EPS@ISEP fosters project-based learning, promotes multicultural and interdisciplinary teamwork, and ethics- and sustainability-driven design. This study applies Natural Language Processing techniques, specifically text mining, to analyse project papers produced by EPS@ISEP teams. The proposed method aims to identify evidence of ethical concerns within EPS@ISEP projects. An innovative keyword mapping approach is introduced that first defines and refines a list of ethics-related keywords through prompt engineering. This enriched list of keywords is then used to systematically map the content of project papers. The findings indicate that the EPS@ISEP robotics project papers analysed demonstrate awareness of ethical considerations and actively incorporate them into design processes. The method presented is adaptable to various application areas, such as monitoring compliance with responsible innovation or sustainability policies.

2026

Enhancing multi-agent deep reinforcement learning for flexible job-shop scheduling through constraint programming

Authors
Jesus, A; Corrêa, A; Vieira, M; Marques, C; Silva, C; Moniz, S;

Publication
COMPUTERS & OPERATIONS RESEARCH

Abstract
This paper introduces PRISMA, a hybrid multi-agent Deep Reinforcement Learning (DRL) framework for solving the Flexible Job-shop Scheduling Problem (FJSP). It uses Constraint Programming (CP) solutions to pretrain decentralized policies and to guide exploration during training. Although DRL can generate fast solutions for large combinatorial problems, it often fails to match the quality of optimization methods, motivating the integration with hybrid frameworks. The growing interest in embedding domain knowledge into learning algorithms has produced several hybrid formulations, yet their potential remains underexplored, particularly in multi-agent settings. PRISMA combines supervised and reinforcement learning within a multi-agent framework, where CP solutions are used to (i) learn expert decisions through imitation learning, and (ii) train an auxiliary network that guides DRL training via reward shaping. A shared graph network is adopted for transferring system-level knowledge into machine-level observations, enabling fast and consistent inference from enriched local embed-dings. To the best of our knowledge, PRISMA introduces the first expert-derived guidance mechanism for the FJSP and is among the earliest to apply imitation learning within a multi-agent formulation. By combining both modules, it strengthens the bridge between optimization and learning-based methods, where such dual integrations remain scarce. Experimental results show faster convergence and higher solution quality than state-ofthe-art DRL models. PRISMA achieves an average optimality gap of 6.74%, corresponding to a 50% relative improvement over the single-agent baseline, while reducing inference time. These findings reinforce the value of merging optimization accuracy with the flexibility of multi-agent DRL for efficient scheduling.

  • 59
  • 4485