Software Quality Assurance and the Transformative Role of Machine Learning in Testing

Abstract

Software defects can affect software quality and increase the cost of software maintenance. Machine learning can help identify defective software modules and support software testing. This study compares three machine learning models for software defect prediction using the NASA PROMISE JM1 dataset. The models are Decision Tree, Bagged Trees, and Support Vector Machine (SVM). The dataset contains 13,204 software modules, including 2,103 defective modules and 11,101 clean modules. Exploratory data analysis was performed to examine the class distribution, missing values, software metrics, outliers, and correlations. The models were evaluated using accuracy, precision, recall, F1-score, and the area under the receiver operating characteristic curve (ROC-AUC). Bagged Trees achieved the highest accuracy of 85.19% and the highest ROC-AUC of 0.741. However, its recall was only 8.57%, which means that it failed to identify many defective modules. SVM achieved an accuracy of 83.30%, a recall of 34.05%, an F1-score of 39.34%, and a ROC-AUC of 0.671. Decision Tree achieved an accuracy of 80.72% and a ROC-AUC of 0.620. Among the three models, SVM achieved the highest recall and F1-score. These results show that high accuracy does not always mean that a model can identify defective software modules effectively. Therefore, several performance measures should be considered when evaluating machine learning models for software defect prediction.