Abstract

Abstract
Training large Deep Neural Networks (DNNs) is inherently resource-intensive and time-consuming, driving significant research into high-frequency checkpointing for enhanced fault tolerance. However, the overhead associated with checkpointing can prolong overall training time. State-of-the-art (SOTA) solutions break down the checkpointing operation into smaller phases, such as snapshot and persist, and pipeline these phases with foreground training operations. Additionally, some strategies leverage faster dynamic or persistent memory devices to minimize overheads. Despite these advancements, SOTA methods still struggle to eliminate training stalls, primarily due to two critical factors: (i) a bandwidth-limited PCIe bus and (ii) consistency requirements for copying the entire model state from the GPU. We propose Fragment — a novel checkpointing solution that relaxes model consistency requirements for fault tolerance, strategically divides the model state into multiple independent pieces (referred to as fragments ), decreases checkpointing overheads, and increases checkpointing frequency, all without significantly impacting model accuracy. Partitioning complex models into fragments creates challenges that we address by ensuring the integrity of the model state during both creation and restoration, while optimizing the selection of layers. Fragment reduces checkpointing overheads by 15% to 94% while decreasing the total data checkpointed by 43% to 80%, without compromising fault-tolerance needs compared to SOTA solutions. Fragment can also reduce recovery time by up to 97% upon each failure recovery, in the worst case.

Abstract

Abstract
Effective selection of forecasting algorithms for time series data is a challenge in machine learning, impacting both predictive accuracy and efficiency. Metalearning, using features extracted from time series, offers a strategic approach to optimize algorithm selection. The utility of this approach depends on the amount of information the features contain about the behavior of the algorithms. Although there are several methods for systematic time series feature extraction, they have never been compared. This paper empirically analyzes the performance of each feature extraction method for algorithm selection and its impact on forecasting accuracy. Our study reveals that TSFRESH, TSFEATURES, and TSFEL exhibit comparable performance at algorithm selection accuracy, adeptly capturing time series characteristics essential for accurate algorithm selection. In contrast, Catch22 is found to be less effective for this purpose. In particular, TSFEL is identified as the most efficient method, balancing dimensionality and predictive performance. These findings provide insights for enhancing forecasting accuracy and efficiency through judicious selection of meta-feature extractors.

Abstract
Predictive Maintenance (PdM) aims to prevent failures through early detection, yet lacks explainability to support decision-making. Current PdM models often identify failures, but fail to explain their root causes, especially in real-world scenarios, with complex and limited labeled data. This study proposes an interpretable framework that combines LSTM-based Anomaly Detection with a dual-layered Root Cause Analysis (RCA) based on SHAP attributions. Applied to a real-world dataset, the method detects degradation transitions, tracks failure patterns over time, and provides interpretable information without explicit root cause labels.

Abstract