![Find Ideas, Do Research, Publish Papers, Earn Degree! Charles Ling ( 凌晓峰 ) , Professor PhD (U of Pennsylvania) University of Western Ontario , Canada ( 加拿大 西安大略大学 ) [email_address] http://cling.csd.uwo.ca](https://image.slidesharecdn.com/howtoresearch-090514020505-phpapp02/85/How-To-Research-1-320.jpg)
![How to Find Research Topics and Do Good Research Charles Ling ( 凌晓峰 ) , Professor PhD (U of Pennsylvania) University of Western Ontario , Canada ( 加拿大 西安大略大学 ) [email_address] http://cling.csd.uwo.ca](https://image.slidesharecdn.com/howtoresearch-090514020505-phpapp02/85/How-To-Research-2-320.jpg)
![Finding Research Ideas The “attitude” when you read papers 拜读，跟踪 30% understanding, 70% critical/creative thinking What is wrong? [ Example ] What else? Why not? How to do better? Many possibilities (creativity – no unique correct answer) From small to major extension “ Surprising effect” (simple but creative) [ Example ] High impact: affecting future research & applications NO !!!](https://image.slidesharecdn.com/howtoresearch-090514020505-phpapp02/85/How-To-Research-7-320.jpg)
![Research Methodologies Prof. Charles X. Ling PhD (U of Pennsylvania) University of Western Ontario, Canada ( 西安大略大学 ) [email_address] http://csd.uwo.ca/faculty/cling](https://image.slidesharecdn.com/howtoresearch-090514020505-phpapp02/85/How-To-Research-32-320.jpg)
![Becoming a Researcher & Research Methodologies Professor Charles X. Ling PhD (U of Pennsylvania) University of Western Ontario, Canada [email_address] http://csd.uwo.ca/faculty/cling](https://image.slidesharecdn.com/howtoresearch-090514020505-phpapp02/85/How-To-Research-33-320.jpg)
How To Research
- 1. Find Ideas, Do Research, Publish Papers, Earn Degree! Charles Ling ( 凌晓峰 ) , Professor PhD (U of Pennsylvania) University of Western Ontario , Canada ( 加拿大 西安大略大学 ) [email_address] http://cling.csd.uwo.ca
- 2. How to Find Research Topics and Do Good Research Charles Ling ( 凌晓峰 ) , Professor PhD (U of Pennsylvania) University of Western Ontario , Canada ( 加拿大 西安大略大学 ) [email_address] http://cling.csd.uwo.ca
- 3. Beginner: re-implement a recent, published work How to expand research… area itself may have not have breakthrough
- 4. To Become a Researcher Research career Scientific discovery Not commercialization Different from engineering The life as a researcher – are you suitable Creative & critical, focus, diligent, persistent, … Graduate students PhD researcher (large companies) or professor (universities)
- 5. First Step: MSc/PhD Candidates You are becoming a researcher (with support) Finding research topics and ideas Doing good research Writing and publishing papers (peer-reviewed) Earning MSc/PhD thesis; applying for grant Two things help young researchers to start Internet Anonymous authorship in paper submission Probably best to obtain PhD in North America
- 6. 1. Finding Research Ideas (Theoretical) Research: discover new knowledge New, sound, significant and better, high impact (Applied research: killer applications) Read latest proceedings and journals Internet makes things much easy Google Scholar, authors’ homepage, email authors www.videolectures.net Attend latest conferences, chat with people Relax: walk on beach, drink some beer…
- 7. Finding Research Ideas The “attitude” when you read papers 拜读，跟踪 30% understanding, 70% critical/creative thinking What is wrong? [ Example ] What else? Why not? How to do better? Many possibilities (creativity – no unique correct answer) From small to major extension “ Surprising effect” (simple but creative) [ Example ] High impact: affecting future research & applications NO !!!
- 8. Proper Model Selection with Significance Test (ECML’08) Abstract: … A common practice is to use the same evaluation metric as the goal one. However, in several recent studies, it is claimed that … In this paper, we point out a potential problem in the experiment design of those studies, and propose an improved method to test the claim. Our extensive experiments show convincingly that only the goal metric itself can most reliably select the correct models. Introduction: … We hope that with the proper model selection method proposed here, we can settle this controversial issue once for all. Back
- 9. When does Co-Training Work in Real Data? (PAKDD09) Abstract : … However, little work has been done on how these assumptions can be empirically verified given datasets. In this paper, we first propose novel approaches to verify empirically … We then propose simple heuristic to split a single view of attributes into two views, and discover regularity on … Our empirical results not only coincide well with the previous theoretical findings, but also provide a practical guideline to decide when co-training should work well based on datasets. Back
- 10. Finding Research Ideas Reading group: read abstract, then brainstorm What is wrong? What would I do? How would I do better? Different levels of discussions Taking notes on new ideas; do not criticize
- 11. From Ideas to Thesis Topics Start with some small and interesting ideas Expand to a broad topic with related problems Both supervisor and student play a role Pay attention to “hot topics”: invited speech in conferences, special issues of journals Make a plan (like a thesis outline) Be flexible with the plan Stay focus with the plan
- 12. An Example: Measures for ML Review: previous work on measure s AUC: a single number measure - C riteria for comparing measures - T heorems for AUC and accuracy - Experimental verification Optimizing learning algorithms with AUC - AUC oriented decision trees Comparing AUC with o ther measures Constructing new measures Discussions and Conclusions
- 13. An Example: Measures for ML Review: previous measure - accuracy , AUC, ROC - measures for orders - profit, cost sensitive learning - Recall, precision, combination - many others in NL retrieval , engineering AUC: a single number measure - criteria for comparing measures - prove theorems for AUC and accuracy (IJCAI’03) - relation between C and D (5 % measures) **** - Experimental verification (CAI’03) - Comparing algorithms with AUC (ICDM’03) - AUC vs profit: **** (TKDE05)
- 14. Optimizing AUC *** - AUC oriented decision trees - AUC oriented SVM - AUC oriented neural networks (submitted) - General theorems about optimizing AUC Other measures and comparison **** - (new) Super AUC measure (sub ’06) - compare and connect others measures - rank measures (ECML’05) - Proof or experiments with other measures (NL, …) Discussions and Conclusions
- 15. 2. Doing Good Research Not an easy task: must make new contributions Know well the state-of-the-art (write a survey paper) Many people may have tried your (new) ideas but failed (not usually published) Usually take a great effort to get it to work Must be very diligent and thorough Implement yours and others’ algorithms Develop new theory and run many experiments Very from different aspects: how others challenge me?
- 16. 3. Writing and Publishing Papers Conferences papers: yearly, so fast, with submission deadline Journal papers: archive, so more complete Dealing with rejection and major revision Paper writing (another talk) English is not our worst enemy Convincing results, clear presentation Work with your supervisor…
- 17. Convincing: Logic of a paper: Problem X is important Previous work A, B, … have been done, but they have certain weaknesses We propose a new method Z… We conduct experiments comparing Z to A and B, and show Z is better Why is Z better? Why didn’t C, D work? What are strengths and weaknesses of Z Conclusions and future work of Z Charles X. Ling
- 18. Clarity: Structure of a Paper Title Abstract Introduction (Review of Previous Work) Our Work Experiments and Comparisons Relation to Previous Work Discussions Conclusions References, Appendix Charles X. Ling A 200-word paper: very high level. Emphasize contributions and significance. Omit details and avoid technical terms A 2-page paper: high level. Emphasize background and motivation A 200-word paper: Summary & future work A 10-word paper Expand on various parts (also top-down structure)
- 19. 6 Typical Mistakes in Paper Writing My paper must be hard to understand I must be formal My paper is big and perfect I am modest/the greatest English is my only problem Reviewers are evil (A separate topic)
- 20. 4. Towards a PhD Thesis… Start early: survey the area, design a plan Submit at least 2 conf papers and 1 journal paper each year PhD thesis: a collection of these papers!
- 21. 2002.9 2006.6 PKDD 03 IJCAI 04 ICML 04 KDD 04 CAI 03 ICML 05 IJCAI 05 On-line system IEEE TKDE JMLR 04 Survey IJCAI 95-02 JMLR 95-02 KDD 95-02 ECML 95-02 AAAI 95-02 MLJ 95-02 JMLR 95-02 Survey paper PhD Thesis
- 22. ( Back )
- 23. To Apply PhD in North America Review carefully requirements on website Most Canadian universities need: Good marks on transcripts – background Good English test score – English Strong recommendation letters – research Usually no GRE is needed Don’t be afraid of asking questions Make your applications as strong as possible Show your research abilities
- 24. Applied Research/Building Systems Find a killer application!!! Keep in mind the grand vision and end users Divide and conquer: sub goals and milestones Application-driven research: new algorithms, exp, scaling up, user study, … Still publishing, and allowing others to use your system Should you worry about being copied? In general, no. You are always the best expert, one-step ahead For research purpose only, getting feedbacks You gain reputation and recognition You may want to start up a new company !
- 25. Summary Research: new, significant, high-impact Msc and PhD candidates: supervised researchers Think critically and creatively Start with small idea, survey the area, build it up Thesis: have a plan early and focus on it Publish yearly in best conferences and journals Applied research: killer applications Internet and anonymous authorship make it fair Researchers/professors are a great career
- 26. Wish you success in research!! Thanks!
- 27. Advance Data Mining and Applications Prof. Charles X. Ling PhD (U of Pennsylvania) University of Western Ontario, Canada ( 西安大略大学 ) [email protected]; [email protected] http://cling/csd.uwo.ca
- 28. Course Plan Wed: Charles Ling, WEKA algorithms and software Thurs: Qiang Yang, DM algorithms and applications Fri: Qiang Yang, DM algorithms and applications Monday: Charles Ling/Qiang Yang: Applications Tues: Wrap up and exam!
- 29. WEKA Textbook: Data Mining: Practical Machine Learning Tools and Techniques (Second Edition), by Morgan Kaufmann, 2005. Software (v4.11): http://www.cs.waikato.ac.nz/ml/weka/ Lecture notes
- 30. Keys in DM Applications Many simple algorithms work quite well Crucial: converting real-world problems into DM problems Use WEKA directly Modify/improve WEKA DM: a lot of science and a bit of art Must understanding the working of algorithms
- 32. Research Methodologies Prof. Charles X. Ling PhD (U of Pennsylvania) University of Western Ontario, Canada ( 西安大略大学 ) [email_address] http://csd.uwo.ca/faculty/cling
- 33. Becoming a Researcher & Research Methodologies Professor Charles X. Ling PhD (U of Pennsylvania) University of Western Ontario, Canada [email_address] http://csd.uwo.ca/faculty/cling
- 34. What Researchers/Scientists do? Create new ideas, invent, discover… Potentially useful in a long run Not merely engineering or applications Not for short term profit (not products)
- 35. Research Support Mostly government grants and funding (NSF, NSERC, NASA, etc.) Based purely on merit of your research
- 36. Research Environment University and research institutes Teaching is light Graduate teaching is useful for research Academic freedom: you are your own “boss” Supported mostly by government funding Tenure system: a lot of free time to explore Supervise graduate students Publish papers
- 37. To Become A Researcher… If you are creative and want to try new ideas… Must be able to focus and think deeply Creativity are important Go to get a Msc/PhD in a good university Becoming a researcher/scientist/professor