Can Great Research Be Taught?(The Unwritten Rules of Research)

Nick Feamster and Alex GrayCollege of Computing

Georgia Tech

Our Story

• An intro to the Ph.D. program existed befores– “Professor parade”: Each professor gave a lecture on

his or her research to all new Ph.D. students– Result: Average student is bored most of the time

• Deans approached us five days before start of Fall 2006 term– “The faculty member who usually teaches this course

has gone on leave. Can you cover it? By the way, we think this course is very broken. You should use this as an opportunity to do whatever you want with it to improve the quality of our students.”

This Talk

• Our experiences designing a course for incoming Ph.D. students at the College of Computing

• Motivation, Course Design, Experiences

• Caveat: Too soon to answer the “big question”– We don’t yet know if great research can be taught– This will only become clear over time as these

students progress through the program

Problem: Underperformance

• Students off to a slow start– Students taking too long to engage in research– Tendency to “drift” in the program without an advisor

• Crisis of confidence– High attrition and drop-out rates in Ph.D. programs– Poor job preparation and marketability– A tendency to blindly follow the advisor’s research

path, rather than blaze new trails

Problem: Missed Opportunities

• The perception that CS is “dead”…– …even as other disciplines are developing an

increasing need for the CS techniques and skills– Untapped potential for impact!

• Missed opportunities for broader social impact

• Perennially low participation by women and minorities

What if we could teach research?

• Goal: Instill in students the skills needed to succeed in a top-tier Ph.D. program– Elements of great research– Elements of great researchers

• Method: Design a syllabus around main skills and concepts (lectures and assignments)

• Underlying Theme: Cross-Disciplinary Thinking

Challenges

• Such a course does not exist– Bits and pieces of “Ph.D. advice” all over the place, but none of it

is amassed in a single place

• Resistance to change– More senior students had taken an old version of the course (the

boring “professor parade”)– We had to battle old perceptions about the course

• “Soft skills” are sometimes viewed as optional• Must appeal to a broad audence: students from all areas

Course Modules Address Obstacles

• Obstacle: How to do research?• Module: How research “works”

– What makes research(ers) great?

• Obstacle: Students lack basic skills/intuition• Module: Research mechanics

• Obstacle: Used to structured environment• Module: Skills for independent work

• Obstacle: Lack of foresight/planning• Module: Career development

Lectures organized around four modules

What Makes Great Research(ers)?

• Good Ideas!– Recognizing: Developing research taste– Generating: Coming up with new ideas (not magic!)– Critiquing: Critically evaluating research– Communicating: Presenting research to the world

• The syllabus includes four major assignments, one for each of these elements

Recognizing Good Ideas

• Assignment 1: Students read top conference proceedings from their research area– Select two “good” papers– Defend their choices

• Course lectures include elements to help students to start developing this taste– “Impact”, risk, blazing new directions, etc.

Generating Good Ideas

• Cross-disciplinary assignment – Read summaries of papers from Assignment 1– Propose an idea that “combines” ideas from some

other area with your own area

• Course includes lectures on idea generation, problem selection, and cross-disciplinary work

• Benefits: Working “at the gap”– Students encouraged to think across boundaries. – Students meet students from outside their area

Communicating Good Ideas• Assignment: Give a presentation to the class

based on a term-long research project with one’s current research advisor

• End-of-term mini-conference

• Work is coupled with students’ RA projects– Ensures that students don’t “blow off” their first term– Option perform this with an additional professor

• Course includes lectures on giving talks and writing papers

Critiquing Ideas

• Assignment: Write a review of other students’ writeups of Assignment 2 (cross-disciplinary ideas)

• Course includes lectures on how to read and review papers

• Benefits: Students learn– Critical evaluation– Subtleties of the review process

Four Mini-Assignments

• Goal: Goal setting (“Why am I here?”)• Assignment: Why Ph.D.?

• Goal: Improve time management skills• Assignment: Time audit log

• Goal: Communication skills and networking• Assignment 1: Research web page• Assignment 2: Multi-resolution elevator pitch

Laying Groundwork

• Career Development– Overview of job opportunities (“professor life”)– Research vs. academic jobs– Teaching and TAing– Personal and research promotion

• Exemplars– External speakers– Examples of broader impact (e.g., social good)

Removing Roadblocks

• Research mechanics– Math skills– Data and analytical skills– Programming– Human-centered skills

• Skills for independent work– Goal setting and motivation– Time management– Information management– Social and personal development

So…Can Great Research Be Taught?

• Too soon to fully evaluate the course’s success• However, can gauge student and faculty

reaction

• Sources of input– Informal feedback– Anonymous surveys in-class– Course evaluation– Town hall meeting– Student mailing lists, etc.– Our observations

Preliminary Observations

• Student attendance was sometimes poor.• Student contribution to group assignments was uneven.• Senior students were quick to criticize the course when

logistical and organizational wrinkles occurred.• Senior students told new ones to selectively attend.• Both students and faculty expressed tension between

mini-projects (which encourage exploration) and term-long projects (which encourage depth).

• We had trouble recruiting high-profile speakers to speak in an introductory course to first-year Ph.D. students.

Biggest Lessons for Us

• Many students lack context to fully appreciate the importance of the course material – “soft” skills and advice contained in lectures, rather than

traditional testable material

• Giving a choice between the mini-projects and a single main project created confusion

• Due to inertia, our radical changes to the course were met with skepticism from older students who had taken the previous version.

Conclusion: Just the Beginning!

• We have developed a first-of-its-kind “Introduction to the CS Ph.D.” course

• Course will form the cornerstone of the program, which strongly encourages– Cross-disciplinary thinking– Broader impact

• The course is still evolving, and the jury is out– We welcome your suggestions for improvements,

evaluation metrics, and related work

Please Contact Us

• Nick Feamsterfeamster@cc.gatech.edu

• Alex Grayagray@cc.gatech.edu

• CS 7001 Course Web Page (Fall 2007)http://www.cc.gatech.edu/classes/AY2008/cs7001_fall/