IterSHAP: An XAI-Based Feature Selection Method for Small High-Dimensional Datasets

Small high-dimensional datasets pose challenges for achieving accurate predictive models, due to issues like overfitting and the curse of dimensionality. While complex models, like deep learning models, have been used to address these challenges, they often lack interpretability and transparency. Explainable Artificial Intelligence (XAI) is a popular field that aims to bridge this gap by developing techniques that provide insights into the decision-making process of machine learning models, therefore increasing the explainability and trustworthiness of models. However, when applying feature selection before model training, less complex models can be used, such that interpretability and explainability are preserved, and explainable AI methods are not even needed. However, in this case, we need an explainable feature selection method. This research presents an iterative feature selection method named IterSHAP, which utilizes a popular XAI technique named SHAP, to increase model performance on small high-dimensional datasets. The performance of IterSHAP was evaluated via both a simulation-based approach and an application-based approach. The results demonstrate the effectiveness of IterSHAP in selecting informative features and improving classification performance on small, high-dimensional datasets.