Abstract

Abstract
Modern supercomputers host numerous jobs that compete for shared storage resources, causing I/O interference and performance degradation. Solutions based on software- defined storage (SDS) emerged to address this issue by coordinating the storage environment through the enforcement of QoS policies. However, these often fail to consider the scale of modern HPC infrastructures.In this work, we explore the advantages and shortcomings of state-of-the-art SDS solutions and highlight the scale of current production clusters and their rising trends. Furthermore, we conduct the first experimental study that sheds new insights into the performance and scalability of flat and hierarchical SDS control plane designs.Our results, using the Frontera supercomputer, show that a flat design with a single controller can scale up to 2,500 nodes with an average control cycle latency of 41 ms, while hierarchical designs can handle up to 10,000 nodes with an average latency ranging between 69 and 103 ms. © 2024 IEEE.

Abstract

Abstract
Ensuring the reproducibility of computational scientific experiments is crucial for advancing research and fostering scientific integrity. However, achieving reproducibility poses significant challenges, particularly in the absence of appropriate software tools to help. This paper addresses this issue by comparing existing tools designed to assist researchers across various fields in achieving reproducibility in their work. We were able to successfully run eight tools and execute them to reproduce three existing experiments from different domains. Our findings show the critical role of technical choices in shaping the capabilities of these tools for reproducibility efforts. By evaluating these tools for replicating experiments, we contribute insights into the current landscape of reproducibility support in scientific research. Our analysis offers guidance for researchers seeking appropriate tools to enhance the reproducibility of their experiments, highlighting the importance of informed technical decisions in facilitating reproducibility across diverse domains.

Abstract
User studies are paramount for advancing science. In particular, the empirical evaluation of programmer-oriented tools is important to validate research ideas and prototypes, as well as production-ready tools. Previous research has collected several tools used by the software engineering and behavioral science communities to design and run studies. In this work, we study tools used in software engineering studies and identify their features. Furthermore, we analyze three behavioral science experiment tools to identify design ideas that might be adapted to programmer user studies. With this work, we present the set of features currently offered by software engineering tools to support researchers in the design and execution of programmer user studies. We also present the characteristics of some tools used in behavioral science experiments to identify design ideas that can be adapted to programmer user studies.

Abstract