Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by HumanISE

2023

A Model for Cognitive Personalization of Microtask Design

Authors
Paulino, D; Guimaraes, D; Correia, A; Ribeiro, J; Barroso, J; Paredes, H;

Publication
SENSORS

Abstract
The study of data quality in crowdsourcing campaigns is currently a prominent research topic, given the diverse range of participants involved. A potential solution to enhancing data quality processes in crowdsourcing is cognitive personalization, which involves appropriately adapting or assigning tasks based on a crowd worker's cognitive profile. There are two common methods for assessing a crowd worker's cognitive profile: administering online cognitive tests, and inferring behavior from task fingerprinting based on user interaction log events. This article presents the findings of a study that investigated the complementarity of both approaches in a microtask scenario, focusing on personalizing task design. The study involved 134 unique crowd workers recruited from a crowdsourcing marketplace. The main objective was to examine how the administration of cognitive ability tests can be used to allocate crowd workers to microtasks with varying levels of difficulty, including the development of a deep learning model. Another goal was to investigate if task fingerprinting can be used to allocate crowd workers to different microtasks in a personalized manner. The results indicated that both objectives were accomplished, validating the usage of cognitive tests and task fingerprinting as effective mechanisms for microtask personalization, including the development of a deep learning model with 95% accuracy in predicting the accuracy of the microtasks. While we achieved an accuracy of 95%, it is important to note that the small dataset size may have limited the model's performance.

2023

Designing for Hybrid Intelligence: A Taxonomy and Survey of Crowd-Machine Interaction

Authors
Correia, A; Grover, A; Schneider, D; Pimentel, AP; Chaves, R; de Almeida, MA; Fonseca, B;

Publication
APPLIED SCIENCES-BASEL

Abstract
With the widespread availability and pervasiveness of artificial intelligence (AI) in many application areas across the globe, the role of crowdsourcing has seen an upsurge in terms of importance for scaling up data-driven algorithms in rapid cycles through a relatively low-cost distributed workforce or even on a volunteer basis. However, there is a lack of systematic and empirical examination of the interplay among the processes and activities combining crowd-machine hybrid interaction. To uncover the enduring aspects characterizing the human-centered AI design space when involving ensembles of crowds and algorithms and their symbiotic relations and requirements, a Computer-Supported Cooperative Work (CSCW) lens strongly rooted in the taxonomic tradition of conceptual scheme development is taken with the aim of aggregating and characterizing some of the main component entities in the burgeoning domain of hybrid crowd-AI centered systems. The goal of this article is thus to propose a theoretically grounded and empirically validated analytical framework for the study of crowd-machine interaction and its environment. Based on a scoping review and several cross-sectional analyses of research studies comprising hybrid forms of human interaction with AI systems and applications at a crowd scale, the available literature was distilled and incorporated into a unifying framework comprised of taxonomic units distributed across integration dimensions that range from the original time and space axes in which every collaborative activity take place to the main attributes that constitute a hybrid intelligence architecture. The upshot is that when turning to the challenges that are inherent in tasks requiring massive participation, novel properties can be obtained for a set of potential scenarios that go beyond the single experience of a human interacting with the technology to comprise a vast set of massive machine-crowd interactions.

2023

A hybrid human-AI tool for scientometric analysis

Authors
Correia, A; Grover, A; Jameel, S; Schneider, D; Antunes, P; Fonseca, B;

Publication
ARTIFICIAL INTELLIGENCE REVIEW

Abstract
Solid research depends on systematic, verifiable and repeatable scientometric analysis. However, scientometric analysis is difficult in the current research landscape characterized by the increasing number of publications per year, intersections between research domains, and the diversity of stakeholders involved in research projects. To address this problem, we propose SciCrowd, a hybrid human-AI mixed-initiative system, which supports the collaboration between Artificial Intelligence services and crowdsourcing services. This work discusses the design and evaluation of SciCrowd. The evaluation is focused on attitudes, concerns and intentions towards use. This study contributes a nuanced understanding of the interplay between algorithmic and human tasks in the process of conducting scientometric analysis.

2023

Investigating Author Research Relatedness through Crowdsourcing: A Replication Study on MTurk

Authors
Correia, A; Paulino, D; Paredes, H; Guimarães, D; Schneider, D; Fonseca, B;

Publication
26th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2023, Rio de Janeiro, Brazil, May 24-26, 2023

Abstract

2023

NLP-Crowdsourcing Hybrid Framework for Inter-Researcher Similarity Detection

Authors
Correia, A; Guimaraes, D; Paredes, H; Fonseca, B; Paulino, D; Trigo, L; Brazdil, P; Schneider, D; Grover, A; Jameel, S;

Publication
IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS

Abstract
Visualizing and examining the intellectual landscape and evolution of scientific communities to support collaboration is crucial for multiple research purposes. In some cases, measuring similarities and matching patterns between research publication document sets can help to identify people with similar interests for building research collaboration networks and university-industry linkages. The premise of this work is assessing feasibility for resolving ambiguous cases in similarity detection to determine authorship with natural language processing (NLP) techniques so that crowdsourcing is applied only in instances that require human judgment. Using an NLP-crowdsourcing convergence strategy, we can reduce the costs of microtask crowdsourcing while saving time and maintaining disambiguation accuracy over large datasets. This article contributes a next-gen crowd-artificial intelligence framework that used an ensemble of term frequency-inverse document frequency and bidirectional encoder representation from transformers to obtain similarity rankings for pairs of scientific documents. A sequence of content-based similarity tasks was created using a crowd-powered interface for solving disambiguation problems. Our experimental results suggest that an adaptive NLP-crowdsourcing hybrid framework has advantages for inter-researcher similarity detection tasks where fully automatic algorithms provide unsatisfactory results, with the goal of helping researchers discover potential collaborators using data-driven approaches.

2023

The technological physical laboratory to achieve improvements in the quality of learning in epistemic terms

Authors
Pequeno, JT; Fonseca, B; Lopes, JBO;

Publication
INTERNATIONAL JOURNAL OF TECHNOLOGY AND DESIGN EDUCATION

Abstract
This work aims to identify teaching and learning practices in practical classes of Computer Network Technology courses, which promote the use of the Physical Laboratory (PL) as an epistemic tool to improve learning in epistemic terms. Content analysis of Multimodal Narrations (MN) of three classes by two teachers were used. An MN aggregates and organizes the data collected in the PL environment. Based on the results, we infer that the student and the teacher, under certain conditions, use the physical laboratory as an epistemic tool since the physical interactions prove its use and reuse. In addition, this study allows, in the context of work in the physical laboratory of networks, to identify that the orchestrations of mediation patterns adopted by the teacher influence the students' epistemic practices and the use of the laboratory as a tool to produce new knowledge. The following contributions are presented: (1) The quality of the students' epistemic practices is increased if, in the teacher's dynamics of mediation, the control of the students' action is reduced; (2) The orchestration of the teacher's mediation patterns is essential to achieve beneficial results in student learning with the use of artifacts from the physical laboratory of Computer Networks; (3) For the physical laboratory to become an epistemic tool, it is necessary that the mediation standards allow students to develop epistemic practices to a high or very high degree and there is a certain mediation orchestration.

  • 28
  • 598