Evaluation of Prediction Accuracy of Classical and AI-Based Software Reliability Models

Abstract

The growing dependence on software-intensive systems across critical application domains has made accurate software reliability prediction a fundamental requirement for ensuring system quality and safety and user confidence. Traditionally, software reliability assessments have relied on classical mathematical and statistical models that estimate failure behavior during the testing phase. Although these approaches have proven useful in relatively stable and controlled environments, their effectiveness often diminishes when applied to modern software systems characterized by increasing complexity, nonlinear interactions, and dynamic operational conditions. These limitations have motivated the exploration of more adaptive and data-driven prediction techniques.

This study presents a comprehensive evaluation of classical software reliability models and artificial intelligence–based machine learning approaches in terms of their prediction accuracy. Conventional statistical models were compared with a range of supervised and ensemble learning algorithms, including k-nearest neighbors, support vector machines, decision trees, random forests, AdaBoost, and gradient boosting. A comparative analysis was conducted using multiple publicly available software reliability datasets, and the model performance was assessed using standard accuracy and error-based evaluation metrics.

The experimental results indicate that AI-driven models consistently achieve higher prediction accuracy and improved robustness across different datasets compared to traditional reliability models. In particular, ensemble learning techniques demonstrate superior performance by effectively capturing complex failure patterns and reducing the prediction variance. Although classical models offer advantages such as simplicity and interpretability, they lack the flexibility required to model the evolving failure behavior of modern software systems. The findings of this study highlight the potential of data-driven AI techniques as reliable and scalable solutions for software reliability prediction, supporting their increasing adoption in modern software quality engineering.