Abstract

Abstract

Abstract
Abstract Some performance-based research funding systems rely on expert-assigned journal rankings to allocate resources and guide research evaluation. In Finland, the JuFo system provides journal rankings, determined by experts who assess journals using available metadata, such as bibliometric indicators, alongside qualitative judgment. While prior work has explored machine learning approaches to approximate these rankings, the recent emergence of large language models (LLMs) offers new possibilities for automated, data-driven evaluation. In this study, we examine how well LLMs can replicate JuFo rankings when given the same structured information available to experts, including citation metrics, disciplinary assignments, and publisher metadata. We systematically compare LLM predictions to expert-assigned JuFo ranks using a confusion-matrix analysis to identify cases of alignment and deviation. Our research addresses two key questions: (1) how accurately LLMs estimate journal rankings, and (2) in which situations their predictions diverge from expert judgments and which factors explain these discrepancies. Our findings show that LLMs approximate expert-assigned rankings with high overall accuracy, with most errors occurring between adjacent levels. However, their performance varies systematically across disciplines, and they tend to under-predict top-tier journals, particularly in social sciences and humanities fields.

Abstract

Abstract
Abstract Misinformation poses one of the most urgent challenges of our society and raises the question of how to deal with it and manage its rapid spread. To address this problem, a promising approach relies on AI-based misinformation detection. This chapter of the book offers a critical analysis of the ethical implications associated with the design, deployment, and use of misinformation detectors (MDs). Designing and deploying an MD—an AI system that automatically identifies misinformation—is a complex undertaking that requires an interdisciplinary approach, as the challenges faced by MD designers and deployers encompass not only technical aspects, but also linguistic, sociological, political, and especially ethical dimensions. Our analysis is ethics-oriented and follows two main lines of inquiry: (1) Ethics by Design, which focuses on issues related to the design process of an MD, and (2) Ethics of Impact, which addresses the intended and unintended effects of MD deployment and use.

Abstract