Investigate the work done by others
Good starting point, but rarely current, and usually don’t cover very advanced topics
Rapid communication of results (months between submission and presentation)Important for visibility and to establish priority of research topics
Slower communication of resultsMore thorough editorial and review process
Detailed descriptions of work, not peer-reviewed, that provide information not found in publications
Include results equivalent to one or several papers, submitted by a student seeking a degree. Reviewed by a committee of faculty.
Annual conference solicits papers in a particular area (e.g. networking, AI, ML, computer architecture)A submission deadline is announced many months in advanceSubmissions have strict page limitsThe conference program committee arranges multiple reviews (3-6) for each submission and selects the best to form the conference programTypical acceptance rates range from 10-30%Authors give a 15-30 min presentation at the conference; papers appear in the proceedingsThere are some variants, e.g. majority of papers at NIPS and UAI are presented as postersSome areas and conferences are experimenting with different submission and reviewing models (e.g. VLDB, ICML)
Reputation for technical excellenceHigh-quality reviewing and strict acceptance standardsThe most significant and technically strongest new resultsAn audience that includes the well-respected members of the field, which increases the impact of results reported there
Some online rankings also exist, which attempt to be quantitative
There are some who call for CS to “grow up”
The system has memory – the same reviewers review subsequent rounds of submission
Guide to finding CSE literature
What is the problem? Why is it important?To whom is it important?
Who has worked on this stuff before?What is novel about the present work?Does the work have any direct competitors?
NoveltyMethodResult summary
E.g., SIGGRAPH papers always have videos, NIPS papers may have proofs, STOC/FOCS papers are technically just “extended abstracts” with proofs in a “full version”
A multi-agent systems paperA machine learning paperA data mining paperA theory paper
Proof?Analytical performance estimate/model?Coded up? Simulated? Put into hardware?How thorough is the testing?
On simulated data? How realistic is the data?On real data? How much/what kind?In interactive systems, is the algorithm “in the loop”?User studies?
What metrics are reported? Are there error bars or other statistical tests?Comparison to competitors?
New methods?New explanations or a new model for something?Comparison of existing things?New empirical evidence of something?
“This method results in fewer dropped packets under realistic loads”“This method predicts significantly more regulatory motifs than…”“We have developed a new algorithm to substantially increase the accuracy of branch prediction in out-of-order, superscalar processors”
-- How well are these statements supported? Do you believe them?