Label Ranking with Partial Abstention using Ensemble Learning
- 1. Weiwei Cheng & Eyke Hüllermeier Knowledge Engineering & Bioinformatics Lab Department of Mathematics and Computer Science University of Marburg, Germany
- 2. Label Ranking (an example) Learning geeks’ preferences on hotels Golf ≻ Park ≻ Krim label ranking Krim ≻ Golf ≻ Park geek 1 Krim ≻ Park ≻ Golf geek 2 Park ≻ Golf ≻ Krim geek 3 geek 4 new geek ??? where the geek could be described by feature vectors, e.g., (gender, age, place of birth, is a professor, …) 1/12
- 3. Label Ranking (an example) Learning geeks’ preferences on hotels Golf Park Krim geek 1 1 2 3 geek 2 2 3 1 geek 3 3 2 1 geek 4 2 1 3 new geek ? ? ? π(i) = position of the i-th label in the ranking 1: Golf 2: Park 3: Krim 2/12
- 4. Label Ranking (more formally) Given: a set of training instances a set of labels for each training instance : a set of pairwise preferences of the form (for some of the labels) Find: A ranking function ( mapping) that maps each to a ranking of (permutation ) and generalizes well in terms of a loss function on rankings (e.g., Kendall’s tau) 3/12
- 5. Existing Approaches Constraint classification Har-Peled , Roth, and Zimak, NIPS-03 Log linear models for label ranking Dekel, Manning, and Singer, NIPS-03 Label ranking by learning pairwise preferences Hüllermeier, Fürnkranz, Cheng, and Brinker, Artificial Intelligence Decision tree and instance-based learning for label ranking Cheng, Hühn, and Hüllermeier, ICML-09 4/12
- 6. Learning with Reject Option usnews.com To train a learner that is able to say “I don’t know”. laptoplogic.com 5/12
- 7. Label Ranking with Reject Option predict a≻b or b≻a, or For each pair of labels a and b, the learner can abstain from prediction (reject option). The learner should be consistent (transitivity). partial orders 6/12
- 8. Label Ranking Ensemble Create a “committee of experts” en.ce.cn 7/12
- 9. Label Ranking Ensemble ≻1 , ≻2 , ……. , ≻k . For a query, setup a label ranking ensemble of size k Define a partial order with 7/12
- 10. Two Problems If a≻b and b≻c, then a≻c. If a≻b and b≻c, then not c≻a. problem Transitivity No cycle Get transitive closure with solution to be solved Marshall’s algorithm. 8/12
- 11. Proposition Given a set of total orders on a finite set , denote by Pab any triple of elements a, b, c ∈ , we have the proportion of orders in which a precedes b. Then, for Pca ≤ 2 − Pab − Pbc . S.t. (Pab ≥ 2/3) ⋀ (Pbc ≥ 2/3) ⟹ (Pca ≤ 2/3) Choosing t > 2/3, we can guarantee acyclic. 9/12
- 12. Experimental Setting dataset #instance #attribute #labels iris 150 4 3 wine 178 13 3 glass 214 9 6 vowel 528 10 11 vehicle 846 18 4 Evaluation metrics prediction true ranking 10/12
- 13. Experimental Results threshold iris wine glass vowel vehicle original 0.868±0.093 0.884±0.078 0.793±0.070 0.324±0.028 0.809±0.034 0.7 0.919±0.066 0.918±0.079 0.847±0.055 0.436±0.034 0.851±0.032 0.8 0.921±0.064 0.956±0.057 0.869±0.055 0.478±0.039 0.872±0.031 0.9 0.940±0.050 0.971±0.049 0.892±0.054 0.515±0.045 0.896±0.031 1.0 0.950±0.045 0.995±0.019 0.928±0.046 0.563±0.056 0.926±0.027 ensemble size of 10 11/12
- 14. Our Contributions A first attempt on label ranking with reject option; Output a reliably partial ranking with ensemble learning. Follow up! The End
- 15. Google “kebi germany” for more info.