Wrapper feature selection method
Download as PPTX, PDF•1 like•2,006 views
The document discusses the benefits and methods of feature subset selection, emphasizing its role in addressing the curse of dimensionality, improving model performance, and reducing training times. It outlines wrapper methods that evaluate feature subsets through model training, and also describes forward selection and backward elimination approaches. Additionally, it mentions the Relief method for attribute evaluation, noting its limitations in handling redundant features.
Wrapper feature selection method
- 1. Wrappers for feature subset selection Amir Razmjou
- 2. Benefits of Feature Subset Selection • Too many dimensions; Elimination of the curse of dimensionality problem • Improved model and classifier performance • Simple Models and elimination of over- fitting • Faster training times.
- 3. The Process
- 4. Feature Selection Methods Wrapper use a search algorithm to search through the space of possible features and evaluate each subset by running a model on the subset Risk of over fitting to the model Computationally expensive Embedded Embedded in and specific to a model Filter Similar to Wrappers in the search approach Simpler filter is evaluated
- 6. Definition 2
- 8. Definition 3
- 9. Contradiction 3 All features are irrelevant
- 10. Definition 4
- 11. Contradiction 4 every feature is relevant Y=TRUE S1=X1=TRUE S2=X2=TRUE Y= X1 X2 X3 X4 = ! X2 X5 = !X3 X1 XOR X2 FALSE TRUE TRUE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE TRUE TRUE TRUE FALSE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE TRUE TRUE FALSE TRUE TRUE FALSE FALSE TRUE FALSE TRUE FALSE FALSE TRUE FALSE FALSE FALSE TRUE TRUE FALSE FALSE FALSE FALSE FALSE TRUE TRUE
- 12. Definition 5 – Strong Relevance
- 13. Definition 6 – Weak Relevance
- 14. Conclusion 1
- 15. • Forward selection methods: these methods start with one or a few features selected according to a method specific selection criteria. More features are iteratively added until a stopping criterion is met. • Backward elimination methods: methods of this type start with all features and iteratively remove one feature or bunches of features.
- 16. Relief • Evaluates the worth of an attribute by repeatedly sampling an instance and considering the value of the given attribute for the nearest instance of the same and different class. Can operate on both discrete and continuous class data.
- 17. Relief • Relief does not help with redundant features. If most of the given features are relevant to the concept, it would select most of them even though only a fraction are necessary for concept description