Cookies Policy
The website need some cookies and similar means to function. If you permit us, we will use those means to collect data on your visits for aggregated statistics to improve our service. Find out More
Accept Reject
  • Menu
Publications

Publications by Armando Sousa

2021

Visible and Thermal Image-Based Trunk Detection with Deep Learning for Forestry Mobile Robotics

Authors
da Silva, DQ; dos Santos, FN; Sousa, AJ; Filipe, V;

Publication
JOURNAL OF IMAGING

Abstract
Mobile robotics in forests is currently a hugely important topic due to the recurring appearance of forest wildfires. Thus, in-site management of forest inventory and biomass is required. To tackle this issue, this work presents a study on detection at the ground level of forest tree trunks in visible and thermal images using deep learning-based object detection methods. For this purpose, a forestry dataset composed of 2895 images was built and made publicly available. Using this dataset, five models were trained and benchmarked to detect the tree trunks. The selected models were SSD MobileNetV2, SSD Inception-v2, SSD ResNet50, SSDLite MobileDet and YOLOv4 Tiny. Promising results were obtained; for instance, YOLOv4 Tiny was the best model that achieved the highest AP (90%) and F1 score (89%). The inference time was also evaluated, for these models, on CPU and GPU. The results showed that YOLOv4 Tiny was the fastest detector running on GPU (8 ms). This work will enhance the development of vision perception systems for smarter forestry robots.

2020

Automatic Visual Inspection of Turbo Vanes produced by Investment Casting Process

Authors
Costa, V; Cardoso, R; Alves, B; Félix, R; Sousa, A; Reis, A;

Publication
SSIP 2020: 2020 3rd International Conference on Sensors, Signal and Image Processing, Prague, Czech Republic, October 9-11, 2020

Abstract
Visual inspection based systems are important tools to ensure the quality of manufactured parts in industry. This work presents an automatic visual inspection approach for defect detection in turbo vanes in the investment casting industry. The proposed method uses RANSAC for robust line and circle detection to extract relevant information to discriminate between a good part and a defected one. Then, using this data a feature vector is created serving as input to a SVM classifier that after the training phase is able to discriminate and classify between a good sample or not. To test the proposed approach a private database was created containing 650 turbo vanes (which gives 2600 different samples to train and test). On this database the proposed method achieved an average accuracy of 99.96%, an average false negative rate of 0.00% and an average false positive rate of 0.05%, using a 5-fold cross validation protocol, which demonstrates the success of the proposed method. Moreover, the proposed image processing pipeline was deployed into Raspberry Pi 4 Model B part of a visual inspection machine, and is working daily at ZCP-Zollern and Comandita Portugal, which proves the method's robustness. © 2020 Owner/Author.

2021

Autonomous Robot Visual-Only Guidance in Agriculture Using Vanishing Point Estimation

Authors
Sarmento, J; Aguiar, AS; dos Santos, FN; Sousa, AJ;

Publication
PROGRESS IN ARTIFICIAL INTELLIGENCE (EPIA 2021)

Abstract
Autonomous navigation in agriculture is very challenging as it usually takes place outdoors where there is rough terrain, uncontrolled natural lighting, constantly changing organic scenarios and sometimes the absence of a Global Navigation Satellite System (GNSS). In this work, a single camera and a Google coral dev Board Edge Tensor Processing Unit (TPU) setup is proposed to navigate among a woody crop, more specifically a vineyard. The guidance is provided by estimating the vanishing point and observing its position with respect to the central frame, and correcting the steering angle accordingly. The vanishing point is estimated by object detection using Deep Learning (DL) based Neural Networks (NN) to obtain the position of the trunks in the image. The NN's were trained using Transfer Learning (TL), which requires a smaller dataset than conventional training methods. For this purpose, a dataset with 4221 images was created considering image collection, annotation and augmentation procedures. Results show that our framework can detect the vanishing point with an average of the absolute error of 0.52. and can be considered for autonomous steering.

2021

Robot navigation in vineyards based on the visual vanish point concept

Authors
Sarmento, J; Aguiar, AS; Santos, FND; Sousa, AJ;

Publication
2021 International Symposium of Asian Control Association on Intelligent Robotics and Industrial Automation, IRIA 2021

Abstract
Autonomous navigation in agriculture is very challenging as it usually takes place outdoors where there is rough terrain, uncontrolled natural lighting, constantly changing organic scenarios and sometimes the absence of Global Navigation Satellite System (GNSS) signal. In this work, a monocular visual system is proposed to estimate angular orientation and navigate between woody crops, more specifically a vineyard, using a Proportional Integrative Derivative (PID)-based controller. The guidance is provided by combining two ways to find the center of the vineyard: First, by estimating the vanishing point and second, by averaging the position of the two closest base trunk detections. Then, by the monocular angle perception, the angular error is determined. For obtaining the trunk position in the image, object detection using Deep Learning (DL) based Neural Networks (NN) is used. To evaluate the proposed controller, a visual vineyard simulation is created using Gazebo. The proposed joint controller is able to travel along a simulated straight vineyard with an RMS error of 1.17 cm. Moreover, a simulated curved vineyard modeled after the Douro region is tested in this work, where the robot was able to steer with an RMS error of 7.28 cm. © 2021 IEEE.

2021

Unimodal and Multimodal Perception for Forest Management: Review and Dataset

Authors
da Silva, DQ; dos Santos, FN; Sousa, AJ; Filipe, V; Boaventura Cunha, J;

Publication
COMPUTATION

Abstract
Robotics navigation and perception for forest management are challenging due to the existence of many obstacles to detect and avoid and the sharp illumination changes. Advanced perception systems are needed because they can enable the development of robotic and machinery solutions to accomplish a smarter, more precise, and sustainable forestry. This article presents a state-of-the-art review about unimodal and multimodal perception in forests, detailing the current developed work about perception using a single type of sensors (unimodal) and by combining data from different kinds of sensors (multimodal). This work also makes a comparison between existing perception datasets in the literature and presents a new multimodal dataset, composed by images and laser scanning data, as a contribution for this research field. Lastly, a critical analysis of the works collected is conducted by identifying strengths and research trends in this domain.

2022

Localization and Mapping on Agriculture Based on Point-Feature Extraction and Semiplanes Segmentation From 3D LiDAR Data

Authors
Aguiar, AS; dos Santos, FN; Sobreira, H; Boaventura Cunha, J; Sousa, AJ;

Publication
FRONTIERS IN ROBOTICS AND AI

Abstract
Developing ground robots for agriculture is a demanding task. Robots should be capable of performing tasks like spraying, harvesting, or monitoring. However, the absence of structure in the agricultural scenes challenges the implementation of localization and mapping algorithms. Thus, the research and development of localization techniques are essential to boost agricultural robotics. To address this issue, we propose an algorithm called VineSLAM suitable for localization and mapping in agriculture. This approach uses both point- and semiplane-features extracted from 3D LiDAR data to map the environment and localize the robot using a novel Particle Filter that considers both feature modalities. The numeric stability of the algorithm was tested using simulated data. The proposed methodology proved to be suitable to localize a robot using only three orthogonal semiplanes. Moreover, the entire VineSLAM pipeline was compared against a state-of-the-art approach considering three real-world experiments in a woody-crop vineyard. Results show that our approach can localize the robot with precision even in long and symmetric vineyard corridors outperforming the state-of-the-art algorithm in this context.

  • 12
  • 21