Feature Selection for high-dimensional data: the issue of stability

Feature selection has become a necessary step to the analysis of high-dimensional datasets coming from several application domains (e.g., web data, document and image analysis, biological data). Though well-established methods exist to select highly discriminative features, discarding the ones that may be either redundant or irrelevant to the problem at hand, little attention has been so far given to the stability of these methods, in cases where the composition of the original dataset is perturbed to some extent (e.g., by adding new records or by random sampling). In this work, we highlight the importance of jointly considering both stability and predictive performance when the selection results are used for knowledge discovery and domain understanding. As a case study, we consider five popular feature selection algorithms, representatives of different selection approaches, and experimentally investigate their behaviour across three different domains: Internet advertisements, text categorization and biomedical data classification. Useful insight on the “intrinsic” stability of each algorithm seems to emerge, despite the peculiar characteristics of each dataset.