Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by Jaime Cardoso

2019

On the role of multimodal learning in the recognition of sign language

Authors
Ferreira, PM; Cardoso, JS; Rebelo, A;

Publication
MULTIMEDIA TOOLS AND APPLICATIONS

Abstract
Sign Language Recognition (SLR) has become one of the most important research areas in the field of human computer interaction. SLR systems are meant to automatically translate sign language into text or speech, in order to reduce the communicational gap between deaf and hearing people. The aim of this paper is to exploit multimodal learning techniques for an accurate SLR, making use of data provided by Kinect and Leap Motion. In this regard, single-modality approaches as well as different multimodal methods, mainly based on convolutional neural networks, are proposed. Our main contribution is a novel multimodal end-to-end neural network that explicitly models private feature representations that are specific to each modality and shared feature representations that are similar between modalities. By imposing such regularization in the learning process, the underlying idea is to increase the discriminative ability of the learned features and, hence, improve the generalization capability of the model. Experimental results demonstrate that multimodal learning yields an overall improvement in the sign recognition performance. In particular, the novel neural network architecture outperforms the current state-of-the-art methods for the SLR task.

2019

Machine Learning Interpretability: A Survey on Methods and Metrics

Authors
Carvalho, DV; Pereira, EM; Cardoso, JS;

Publication
ELECTRONICS

Abstract
Machine learning systems are becoming increasingly ubiquitous. These systems's adoption has been expanding, accelerating the shift towards a more algorithmic society, meaning that algorithmically informed decisions have greater potential for significant social impact. However, most of these accurate decision support systems remain complex black boxes, meaning their internal logic and inner workings are hidden to the user and even experts cannot fully understand the rationale behind their predictions. Moreover, new regulations and highly regulated domains have made the audit and verifiability of decisions mandatory, increasing the demand for the ability to question, understand, and trust machine learning systems, for which interpretability is indispensable. The research community has recognized this interpretability problem and focused on developing both interpretable models and explanation methods over the past few years. However, the emergence of these methods shows there is no consensus on how to assess the explanation quality. Which are the most suitable metrics to assess the quality of an explanation? The aim of this article is to provide a review of the current state of the research field on machine learning interpretability while focusing on the societal impact and on the developed methods and metrics. Furthermore, a complete literature review is presented in order to identify future directions of work on this field.

2018

mu SmartScope: Towards a Fully Automated 3D-Printed Smartphone Microscope with Motorized Stage

Authors
Rosado, L; Silva, PT; Faria, J; Oliveira, J; Vasconcelos, MJM; Elias, D; da Costa, JMC; Cardoso, JS;

Publication
BIOMEDICAL ENGINEERING SYSTEMS AND TECHNOLOGIES (BIOSTEC 2017)

Abstract
Microscopic examination is the reference diagnostic method for several neglected tropical diseases. However, its quality and availability in rural endemic areas is often limited by the lack of trained personnel and adequate equipment. These drawbacks are closely related with the increasing interest in the development of computer-aided diagnosis systems, particularly distributed solutions that provide access to complex diagnosis in rural areas. In this work we present our most recent advances towards the development of a fully automated 3D-printed smartphone microscope with a motorized stage, termed mu SmartScope. The developed prototype allows autonomous acquisition of a pre-defined number of images at 1000x magnification, by using a motorized automated stage fully powered and controlled by a smartphone, without the need of manual focus. In order to validate the prototype as a reliable alternative to conventional microscopy, we evaluated the mu SmartScope performance in terms of: resolution; field of view; illumination; motorized stage performance (mechanical movement precision/resolution and power consumption); and automated focus. These results showed similar performances when compared with conventional microscopy, plus the advantage of being low-cost and easy to use, even for non-experts in microscopy. To extract these results, smears infected with blood parasites responsible for the most relevant neglected tropical diseases were used. The acquired images showed that it was possible to detect those agents through images acquired via the mu SmartScope, which clearly illustrate the huge potential of this device, specially in developing countries with limited access to healthcare services.

2019

A Single-Resolution Fully Convolutional Network for Retinal Vessel Segmentation in Raw Fundus Images

Authors
Araujo, RJ; Cardoso, JS; Oliveira, HP;

Publication
IMAGE ANALYSIS AND PROCESSING - ICIAP 2019, PT II

Abstract
The segmentation of retinal vessels in fundus images has been heavily focused in the past years, given their relevance in the diagnosis of several health conditions. Even though the recent advent of deep learning allowed to foster the performance of computer-based algorithms in this task, further improvement concerning the detection of vessels while suppressing background noise has clinical significance. Moreover, the best performing state-of-the-art methodologies conduct patch-based predictions. This, put together with the preprocessing techniques used in those methodologies, may hinder their use in screening scenarios. Thus, in this paper, we explore a fully convolutional setting that takes raw fundus images and allows to combine patch-based training with global image prediction. Our experiments on the DRIVE, STARE and CHASEDB1 databases show that the proposed methodology achieves state-of-the-art performance in the first and the last, allowing at the same time much faster segmentation of new images.

2019

Automatic Augmentation by Hill Climbing

Authors
Cruz, R; Costa, JFP; Cardoso, JS;

Publication
ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2019: DEEP LEARNING, PT II

Abstract
When learning from images, it is desirable to augment the dataset with plausible transformations of its images. Unfortunately, it is not always intuitive for the user how much shear or translation to apply. For this reason, training multiple models through hyperparameter search is required to find the best augmentation policies. But these methods are computationally expensive. Furthermore, since they generate static policies, they do not take advantage of smoothly introducing more aggressive augmentation transformations. In this work, we propose repeating each epoch twice with a small difference in data augmentation intensity, walking towards the best policy. This process doubles the number of epochs, but avoids having to train multiple models. The method is compared against random and Bayesian search for classification and segmentation tasks. The proposal improved twice over random search and was on par with Bayesian search for 4% of the training epochs.

2019

Deep Vesselness Measure from Scale-Space Analysis of Hessian Matrix Eigenvalues

Authors
Araújo, RJ; Cardoso, JS; Oliveira, HP;

Publication
PATTERN RECOGNITION AND IMAGE ANALYSIS, IBPRIA 2019, PT II

Abstract
The enhancement of tubular structures such as vessels in medical images has been addressed in the past, aiming for easier extraction and or visualization of such structures by professionals. Some literature methodologies propose vesselness measures whose design is motivated by local properties of vascular networks and how these influence the eigenvalues of the Hessian matrix. However, past work fails to combine properly the scale-space and neighborhood information, thus leading to the proposal of suboptimal vesselness measures. In this paper, we show that a shallow convolutional neural network is able to learn more optimal embedding spaces from the eigenvalue analysis at different scales, thus leading to a stronger vessel enhancement. Additionally, we also show that such a system maintains one of the biggest advantages of Hessian-based vesselness measures, which is the robustness to data with varying statistics. © 2019, Springer Nature Switzerland AG.

  • 19
  • 60