Authors: Bilal Ibrahim Maijamaa, Salim Ahmad, Zaharaddeen Salele Iro, Aminu Aliyu Abdullahi
Abstract: Missing values are a common challenge in real-world datasets, often reducing the accuracy and reliability of machine learning models. Although numerous machine learning and ensemble-based imputation techniques have been proposed, many rely on single-level stacking architectures and do not explicitly model both linear and nonlinear relationships during prediction. This study proposes a Particle Swarm Optimization (PSO)-optimized two-level stacking ensemble for numerical data imputation. The framework employs PSO for base-learner selection and hyperparameter optimization; the selected base learners are Random Forest and XGBoost. A two-metalearner architecture is then used in which Linear Regression captures linear dependencies and Random Forest models nonlinear interactions to generate the final imputed values. The proposed framework was evaluated on the Breast Cancer Wisconsin and Wine Quality datasets under Missing Completely at Random (MCAR) mechanisms at 30%, 20%, and 10% missingness. Performance was assessed using Root Mean Square Error (RMSE), Mean Absolute Error (MAE), coefficient of determination (R²), and processing time. Experimental results demonstrate that the proposed model consistently outperformed the standalone Random Forest and XGBoost models across all missingness levels. On the Breast Cancer dataset, the proposed model achieved RMSE values of 0.0747, 0.0627, and 0.0529 with corresponding R² values of 67.66%, 70.51%, and 74.63% at 30%, 20%, and 10% MCAR, respectively. Similarly, on the Wine Quality dataset, it recorded RMSE values of 0.1873, 0.1751, and 0.1681 with corresponding R² values of 49.54%, 52.15%, and 54.64%. Furthermore, the proposed approach outperformed a recently reported hybrid imputation method, achieving substantially lower prediction errors while maintaining acceptable computational cost. These findings demonstrate that integrating PSO optimization with a two-level stacking ensemble provides an accurate, robust, and scalable framework for numerical missing value imputation across datasets with varying degrees of missingness.
