how to research

Find Ideas, Do Research, Publish Papers, Earn Degree!

Charles Ling ( 凌晓峰 ), ProfessorPhD (U of Pennsylvania)

University of Western Ontario, Canada( 加拿大 西安大略大学 )

cling@csd.uwo.ca

http://cling.csd.uwo.ca

Charles X. Ling

To Become a Researcher To Become a Researcher

Research careerScientific discovery

– Not commercialization – Different from engineering

The life as a researcher – are you suitable– Creative & critical, focus, diligent, persistent, …

Graduate students PhD researcher (large companies) or professor (universities)

Charles X. Ling

First Step: MSc/PhD CandidatesFirst Step: MSc/PhD CandidatesYou are becoming a researcher (with support)

1. Finding research topics and ideas2. Doing good research3. Writing and publishing papers (peer-reviewed)4. Earning MSc/PhD thesis; applying for grant

Two things help young researchers to start– Internet– Anonymous authorship in paper submission

Probably best to obtain PhD in North America

Charles X. Ling

1. Finding Research Ideas1. Finding Research Ideas(Theoretical) Research: discover new knowledge

– New, sound, significant and better, high impact (Applied research: killer applications) Read latest proceedings and journalsInternet makes things much easy

– Google Scholar, authors’ homepage, email authors– www.videolectures.net

Attend latest conferences, chat with peopleRelax: walk on beach, drink some beer…

Charles X. Ling

Finding Research IdeasFinding Research IdeasThe “attitude” when you read papers

– 拜读，跟踪 30% understanding, 70% critical/creative thinking

– What is wrong? [Example]– What else? Why not? How to do better?

Many possibilities (creativity – no unique correct answer) From small to major extension “Surprising effect” (simple but creative) [Example] High impact: affecting future research & applications

Charles X. Ling

Finding Research IdeasFinding Research IdeasReading group: read abstract, then brainstorm

– What is wrong?– What would I do?– How would I do better?

Different levels of discussions Taking notes on new ideas; do not criticize

Charles X. Ling

From Ideas to Thesis TopicsFrom Ideas to Thesis TopicsStart with some small and interesting ideasExpand to a broad topic with related problemsBoth supervisor and student play a rolePay attention to “hot topics”: invited speech in

conferences, special issues of journalsMake a plan (like a thesis outline)

– Be flexible with the plan– Stay focus with the plan

Charles X. Ling

An Example: Measures for MLAn Example: Measures for MLReview: previous measure

- accuracy, AUC, ROC - measures for orders - profit, cost sensitive learning - Recall, precision, combination- many others in NL retrieval, engineering

AUC: a single number measure - criteria for comparing measures - prove theorems for AUC and accuracy (IJCAI’03)- relation between C and D (5 % measures) **** - Experimental verification (CAI’03) - Comparing algorithms with AUC (ICDM’03)- AUC vs profit: **** (TKDE05)

Charles X. Ling

Optimizing AUC *** - AUC oriented decision trees - AUC oriented SVM - AUC oriented neural networks (submitted) - General theorems about optimizing AUC

Other measures and comparison **** - (new) Super AUC measure (sub ’06)- compare and connect others measures - rank measures (ECML’05) - Proof or experiments with other measures (NL, …)

Discussions and Conclusions

Charles X. Ling

2. Doing Good Research2. Doing Good ResearchNot an easy task: must make new contributions

– Know well the state-of-the-art (write a survey paper)– Many people may have tried your (new) ideas but

failed (not usually published)– Usually take a great effort to get it to work – Must be very diligent and thorough

Implement yours and others’ algorithms Develop new theory and run many experiments Very from different aspects: how others challenge me?

Charles X. Ling

3. Writing and Publishing Papers3. Writing and Publishing PapersConferences papers: yearly, so fast, with

submission deadline Journal papers: archive, so more complete

– Dealing with rejection and major revision

Paper writing (another talk) – English is not our worst enemy– Convincing results, clear presentation – Work with your supervisor…

Charles X. LingCharles X. Ling

Convincing: Logic of a paper:Convincing: Logic of a paper:– Problem X is important– Previous work A, B, … have been done, but

they have certain weaknesses – We propose a new method Z…– We conduct experiments comparing Z to A

and B, and show Z is better – Why is Z better? Why didn’t C, D work?

What are strengths and weaknesses of Z– Conclusions and future work of Z

Charles X. LingCharles X. Ling

Clarity: Structure of a PaperClarity: Structure of a PaperTitleAbstractIntroduction(Review of Previous Work)Our WorkExperiments and ComparisonsRelation to Previous WorkDiscussionsConclusionsReferences, Appendix

A 200-word paper: very high level. Emphasize contributions and significance. Omit details and avoid technical terms

A 2-page paper: high level. Emphasize background and motivation

A 200-word paper: Summary & future work

Expand on various parts (also top-down structure)

A 10-word paper

Charles X. Ling

4. Towards a PhD Thesis…4. Towards a PhD Thesis…Start early: survey the area, design a planSubmit at least 2 conf papers and 1 journal

paper each yearPhD thesis: a collection of these papers!

Charles X. Ling

PKDD 03

IJCAI 04

ICML 04 KDD 04

CAI 03

ICML 05 IJCAI 05

On-line system IEEE TKDE

JMLR 04

Survey

IJCAI 95-02 JMLR 95-02

KDD 95-02 ECML 95-02

AAAI 95-02

MLJ 95-02 JMLR 95-02 Survey paper

2002.9

PhD Thesis2006.6

Charles X. Ling

Applied Research/Building SystemsApplied Research/Building Systems Find a killer application!!! Keep in mind the grand vision and end users Divide and conquer: sub goals and milestones Application-driven research: new algorithms, exp,

scaling up, user study, … Still publishing, and allowing others to use your system

– Should you worry about being copied? In general, no. You are always the best expert, one-step ahead• For research purpose only, getting feedbacks• You gain reputation and recognition

• You may want to start up a new company !

Charles X. Ling

SummarySummaryResearch: new, significant, high-impactMsc and PhD candidates: supervised researchers Think critically and creatively Start with small idea, survey the area, build it upThesis: have a plan early and focus on itPublish yearly in best conferences and journalsApplied research: killer applications Internet and anonymous authorship make it fair Researchers/professors are a great career

Wish you success in research!!

Thanks!

Charles X. Ling

Advance Data Mining and Advance Data Mining and Applications Applications

Prof. Charles X. LingPhD (U of Pennsylvania)

University of Western Ontario, Canada( 西安大略大学 )

cling@csd.uwo.ca; prof.ling@gmail.com

http://cling/csd.uwo.ca

Charles X. Ling

Course PlanCourse PlanWed: Charles Ling, WEKA algorithms and

softwareThurs: Qiang Yang, DM algorithms and

applications Fri: Qiang Yang, DM algorithms and applications Monday: Charles Ling/Qiang Yang: Applications Tues: Wrap up and exam!

Charles X. Ling

WEKAWEKATextbook: Data Mining: Practical Machine

Learning Tools and Techniques (Second Edition), by Morgan Kaufmann, 2005.

Software (v4.11): http://www.cs.waikato.ac.nz/ml/weka/

Lecture notes

Charles X. Ling

Keys in DM ApplicationsKeys in DM ApplicationsMany simple algorithms work quite wellCrucial: converting real-world problems into

DM problems– Use WEKA directly– Modify/improve WEKA

DM: a lot of science and a bit of art – Must understanding the working of algorithms

Charles X. Ling

Charles X. Ling

Research MethodologiesResearch Methodologies

Prof. Charles X. LingPhD (U of Pennsylvania)

University of Western Ontario, Canada( 西安大略大学 )cling@csd.uwo.ca

http://csd.uwo.ca/faculty/cling

Charles X. Ling

Becoming a Researcher & Becoming a Researcher & Research MethodologiesResearch Methodologies

Professor Charles X. LingPhD (U of Pennsylvania)

University of Western Ontario, Canadacling@csd.uwo.ca

http://csd.uwo.ca/faculty/cling

Charles X. Ling

What Researchers/Scientists do?What Researchers/Scientists do?Create new ideas, invent, discover…

– Potentially useful in a long run– Not merely engineering or applications– Not for short term profit (not products)

Charles X. Ling

Research SupportResearch Support

Mostly government grants and funding (NSF, NSERC, NASA, etc.)

Based purely on merit of your research

Charles X. Ling

Research EnvironmentResearch Environment

University and research institutes Teaching is light

– Graduate teaching is useful for research Academic freedom: you are your own “boss”Supported mostly by government fundingTenure system: a lot of free time to exploreSupervise graduate studentsPublish papers