2018
Authors
da Silva, AR; Paiva, ACR; da Silva, VER;
Publication
Model-Driven Engineering and Software Development - 6th International Conference, MODELSWARD 2018, Funchal, Madeira, Portugal, January 22-24, 2018, Revised Selected Papers
Abstract
Testing is one of the most important activities to ensure the quality of a software system. This paper proposes and discusses the TSL (Test Specification Language) that adopts a model-based testing approach for both human-readable and computer-executable specifications of test cases. TSL is strongly inspired on the grammar, nomenclature and writing style as defined by the RSLingo RSL, which is a rigorous requirements specification language. Both RSL and TSL are controlled natural languages that share common concepts such as data entities, use cases and state machines. However, by applying black-box functional testing design techniques, TSL includes and supports four complementary testing strategies, namely: domain analysis testing; use case tests; state machine testing; and acceptance criteria. This paper focuses on the first three testing strategies of TSL. Finally, a simple but effective case study illustrates the overall approach and supports the discussion. © 2019, Springer Nature Switzerland AG.
2019
Authors
Paiva, ACR; Gouveia, JMEP; Elizabeth, JD; Delamaro, ME;
Publication
2019 IEEE 12TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW 2019)
Abstract
Mobile applications have some specific characteristics not found on web and desktop applications. The mobile testing tools available may not be prepared to detect problems related to those specificities. So, it is important to assess the quality of the test cases generated/executed by mobile testing tools in order to check if they are able to find those specific problems. One way to assess the quality of a test suite is through mutation testing. This paper presents new mutation operators created to inject faults leading to known failures related to the non-preservation of users transient UI state when mobile applications go to background and then come back to foreground. A set of mutation operators is presented and the rational behind its construction is explained. A case study illustrates the approach to evaluate a mobile testing tool. In this study, the tool used is called iMPAcT tool, however any other mobile testing tool could be used. The experiments are performed over mobile applications publicly available on the Google Play store. The results are presented and discussed. Finally, some improvements are suggested for the iMPAcT tool in order to be able to generate test cases that can kill more mutants and so, hopefully, detect more failures in the future.
2019
Authors
Paiva, ACR; Goncalves, MA; Barros, AR;
Publication
2019 IEEE 12TH CONFERENCE ON SOFTWARE TESTING, VALIDATION AND VERIFICATION (ICST 2019)
Abstract
Mobile applications are increasingly present in our daily lives. Being increasingly dependent on apps, we all want to make sure apps work as expected. One way to increase confidence and quality of software is through testing. However, the existing approaches and tools still do not provide sufficient solutions for testing mobile apps with features different from the ones found in desktop or web applications. In particular, there are guidelines that mobile developers should follow and that may be tested automatically but, as far as we know, there are no tools that are able do it. The iMPAcT tool combines exploration, reverse engineering and testing to check if mobile apps follow best practices to implement specific behavior called UI Patterns. Examples of UI Patterns within this catalog are: orientation, background-foreground, side drawer, tab-scroll, among others. For each of these behaviors (UI Patterns), the iMPAcT tool has a corresponding Test Pattern that checks if the UI Pattern implementation follows the guidelines. This paper presents an extension to iMPAcT tool. It enables to test if Android apps work properly after receiving an incoming call, i.e., if the state of the screen after the call is the same as before getting the call. It formalizes the problem, describes the overall approach, describes the architecture of the tool and reports an experiment performed over 61 public mobile apps.
2019
Authors
Morgado, IC; Paiva, ACR;
Publication
Proc. ACM Hum. Comput. Interact.
Abstract
This paper presents iMPAcT tool that tests recurring common behavior on Android mobile applications. The process followed combines exploration, reverse engineering and testing to automatically test Android mobile applications. The tool explores automatically the App by firing UI events. After each event fired, the tool checks if there are UI patterns present using a reverse engineering process. If a UI pattern is present, the tool runs the corresponding testing strategy (Test Pattern). During reverse engineering the tool uses a catalog of UI Patterns which describes recurring behavior (UI Patterns) to test and the corresponding test strategies (Test Patterns). This catalog may be extended in the future as needed (e.g., to deal with new interaction trends). This paper describes the implementation details of the iMPAcT tool, the catalog of patterns used, the outputs produced by the tool and the results of experiments performed in order to evaluate the overall testing approach. These results show that the overall testing approach is capable of finding failures on existing Android mobile applications.
2019
Authors
Maciel, D; Paiva, ACR; da Silva, AR;
Publication
PROCEEDINGS OF THE 14TH INTERNATIONAL CONFERENCE ON EVALUATION OF NOVEL APPROACHES TO SOFTWARE ENGINEERING (ENASE)
Abstract
Frequently software testing tends to be neglected at the beginning of the projects, only performed on the late stage. However, it is possible to benefit from combining requirement with testing specification activities. On one hand, acceptance tests specification will require less manual effort since they are defined or generated automatically from the requirements specification. On the other hand, the requirements specification itself will end up having higher quality due to the use of a more structured language, reducing typical problems such as ambiguity, inconsistency and incorrectness. This research proposes an approach that promotes the practice of tests specification since the very beginning of projects, and its integration with the requirements specification itself. It is a model-driven approach that contributes to maintain the requirements and tests alignment, namely between requirements, test cases, and low-level automated test scripts. To show the applicability of the approach, two complementary languages are adopted: the ITLingo RSL that is particularly designed to support both requirements and tests specification; and the Robot language, which is a low-level keyword-based language for the specification of test scripts. The approach includes model-to-model transformation techniques, such as test cases into test scripts transformations. In addition, these test scripts are executed by the Robot test automation framework.
2019
Authors
Ferreira, J; Paiva, ACR;
Publication
Quality of Information and Communications Technology - 12th International Conference, QUATIC 2019, Ciudad Real, Spain, September 11-13, 2019, Proceedings
Abstract
Smartphones are becoming more important in our everyday lives and it is increasingly common to perform critical tasks on these devices, such as making payments. For this reason, ensuring the quality of these applications is an important task. One way to do this is through software testing. However, the testing of these applications presents major challenges due to the wide variety of devices available in the market. In this context, automated testing gains more relevance. There are dynamic test approaches for testing mobile applications, but there are some challenges that need to be overcome for good results, such as, being able to explore the complete behaviour of the application (e.g., overcoming blocking points); choosing appropriate input data; testing dynamic behaviour; testing specific characteristics of mobile applications, such as specific forms of interaction, e.g., long press, and so on. This paper presents a dynamic exploration approach of Android mobile applications that aims to overcome some of the problems identified. During the exploration process, the algorithm builds a Finite State Machine where states are traversed screens and transitions between states describe events that allow moving from one screen to another. This approach is implemented as an extension of the iMPAcT tool. The approach is validated over real Google Play apps and the test coverage results achieved are presented, compared and discussed. © Springer Nature Switzerland AG 2019.
The access to the final selection minute is only available to applicants.
Please check the confirmation e-mail of your application to obtain the access code.