paper.dvi

Transcript

1 How to Read a Paper S. Keshav Waterloo David R. Cheriton School of Computer Science, University of Waterloo, ON, Canada [email protected] ABSTRACT 4. Glance over the references, mentally ticking off the ones you’ve already read Researchers spend a great deal of time reading research pa- pers. However, this skill is rarely taught, leading to much At the end of the first pass, you should be able to answer wasted effort. This article outlines a practical and efficient : five Cs the for reading research papers. I also de- three-pass method scribe how to use this method to do a literature survey. : What type of paper is this? A measure- Category 1. ment paper? An analysis of an existing system? A Categories and Subject Descriptors: A.1 [Introductory description of a research prototype? and Survey] General Terms: Documentation. Context 2. : Which other papers is it related to? Which Paper, Reading, Hints. Keywords: theoretical bases were used to analyze the problem? Correctness : Do the assumptions appear to be valid? 3. 1. INTRODUCTION Researchers must read papers for several reasons: to re- Contributions 4. : What are the paper’s main contribu- view them for a conference or a class, to keep current in tions? their field, or for a literature survey of a new field. A typi- r cal researcher will likely spend hundreds of hours every yea Clarity 5. : Is the paper well written? reading papers. Using this information, you may choose not to read fur- Learning to efficiently read a paper is a critical but rarely ther. This could be because the paper doesn’t interest you, taught skill. Beginning graduate students, therefore, mus t or you don’t know enough about the area to understand the learn on their own using trial and error. Students waste paper, or that the authors make invalid assumptions. The much effort in the process and are frequently driven to frus- first pass is adequate for papers that aren’t in your research tration. For many years I have used a simple approach to efficiently area, but may someday prove relevant. h Incidentally, when you write a paper, you can expect most read papers. This paper describes the ‘three-pass’ approac reviewers (and readers) to make only one pass over it. Take and its use in doing a literature survey. care to choose coherent section and sub-section titles and to write concise and comprehensive abstracts. If a reviewer THE THREE-PASS APPROACH 2. cannot understand the gist after one pass, the paper will The key idea is that you should read the paper in up to likely be rejected; if a reader cannot understand the high- three passes, instead of starting at the beginning and plow- lights of the paper after five minutes, the paper will likely ing your way to the end. Each pass accomplishes specific never be read. goals and builds upon the previous pass: The first pass gives you a general idea about the paper. The second pass 2.2 The second pass lets you grasp the paper’s content, but not its details. The In the second pass, read the paper with greater care, but third pass helps you understand the paper in depth. ignore details such as proofs. It helps to jot down the key points, or to make comments in the margins, as you read. 2.1 The first pass The first pass is a quick scan to get a bird’s-eye view of 1. Look carefully at the figures, diagrams and other illus- the paper. You can also decide whether you need to do any trations in the paper. Pay special attention to graphs. more passes. This pass should take about five to ten minutes Are the axes properly labeled? Are results shown with and consists of the following steps: error bars, so that conclusions are statistically sig- nificant? Common mistakes like these will separate 1. Carefully read the title, abstract, and introduction rushed, shoddy work from the truly excellent. 2. Read the section and sub-section headings, but ignore 2. Remember to mark relevant unread references for fur- everything else ther reading (this is a good way to learn more about the background of the paper). 3. Read the conclusions ACM SIGCOMM Computer Communication Review 83 Volume 37, Number 3, July 2007

2 That will help you identify the top conferences in that field The second pass should take up to an hour. After this because the best researchers usually publish in the top con- pass, you should be able to grasp the content of the paper. ferences. You should be able to summarize the main thrust of the pa- The third step is to go to the website for these top con- per, with supporting evidence, to someone else. This level o f ferences and look through their recent proceedings. A quick detail is appropriate for a paper in which you are interested , k. scan will usually identify recent high-quality related wor but does not lie in your research speciality. These papers, along with the ones you set aside earlier, con- Sometimes you won’t understand a paper even at the end stitute the first version of your survey. Make two passes of the second pass. This may be because the subject matter through these papers. If they all cite a key paper that you is new to you, with unfamiliar terminology and acronyms. did not find earlier, obtain and read it, iterating as neces- Or the authors may use a proof or experimental technique sary. that you don’t understand, so that the bulk of the pa- per is incomprehensible. The paper may be poorly written with unsubstantiated assertions and numerous forward ref- 4. EXPERIENCE erences. Or it could just be that it’s late at night and you’re I’ve used this approach for the last 15 years to read con- tired. You can now choose to: (a) set the paper aside, hoping h, ference proceedings, write reviews, do background researc you don’t need to understand the material to be successful and to quickly review papers before a discussion. This dis- in your career, (b) return to the paper later, perhaps after ciplined approach prevents me from drowning in the details reading background material or (c) persevere and go on to before getting a bird’s-eye-view. It allows me to estimate t he the third pass. amount of time required to review a set of papers. More- over, I can adjust the depth of paper evaluation depending 2.3 The third pass on my needs and how much time I have. To fully understand a paper, particularly if you are re- viewer, requires a third pass. The key to the third pass 5. RELATED WORK is to attempt to virtually re-implement the paper: that is, making the same assumptions as the authors, re-create the If you are reading a paper to do a review, you should also work. By comparing this re-creation with the actual paper, read Timothy Roscoe’s paper on “Writing reviews for sys- you can easily identify not only a paper’s innovations, but al tems conferences” [1]. If you’re planning to write a technic also its hidden failings and assumptions. paper, you should refer both to Henning Schulzrinne’s com- This pass requires great attention to detail. You should prehensive web site [2] and George Whitesides’s excellent identify and challenge every assumption in every statement . overview of the process [3]. Moreover, you should think about how you yourself would present a particular idea. This comparison of the actual 6. A REQUEST with the virtual lends a sharp insight into the proof and I would like to make this a living document, updating it presentation techniques in the paper and you can very likely as I receive comments. Please take a moment to email me add this to your repertoire of tools. During this pass, you any comments or suggestions for improvement. You can also should also jot down ideas for future work. add comments at CCRo, the online edition of CCR [4]. This pass can take about four or five hours for beginners, and about an hour for an experienced reader. At the end 7. ACKNOWLEDGMENTS of this pass, you should be able to reconstruct the entire structure of the paper from memory, as well as be able to The first version of this document was drafted by my stu- ld identify its strong and weak points. In particular, you shou dents: Hossein Falaki, Earl Oliver, and Sumair Ur Rahman. be able to pinpoint implicit assumptions, missing citation s My thanks to them. I also benefited from Christophe Diot’s to relevant work, and potential issues with experimental or perceptive comments and Nicole Keshav’s eagle-eyed copy- analytical techniques. editing. This work was supported by grants from the National Science and Engineering Council of Canada, the Canada 3. DOING A LITERATURE SURVEY Research Chair Program, Nortel Networks, Microsoft, Intel Paper reading skills are put to the test in doing a literature Corporation, and Sprint Corporation. survey. This will require you to read tens of papers, perhaps in an unfamiliar field. What papers should you read? Here 8. REFERENCES is how you can use the three-pass approach to help. First, use an academic search engine such as Google Scholar [1] T. Roscoe, “Writing Reviews for Systems or CiteSeer and some well-chosen keywords to find three to Conferences,” papers in the area. Do one pass on each pa- five recent http://people.inf.ethz.ch/troscoe/pubs/review- per to get a sense of the work, then read their related work writing.pdf. sections. You will find a thumbnail summary of the recent [2] H. Schulzrinne, “Writing Technical Articles,” work, and perhaps, if you are lucky, a pointer to a recent http://www.cs.columbia.edu/ hgs/etc/writing- survey paper. If you can find such a survey, you are done. style.html. Read the survey, congratulating yourself on your good luck. [3] G.M. Whitesides, “Whitesides’ Group: Writing a Otherwise, in the second step, find shared citations and Paper,” repeated author names in the bibliography. These are the http://www.che.iitm.ac.in/misc/dd/writepaper.pdf. key papers and researchers in that area. Download the key [4] ACM SIGCOMM Computer Communication Review papers and set them aside. Then go to the websites of the Online, http://www.sigcomm.org/ccr/drupal/. key researchers and see where they’ve published recently. ACM SIGCOMM Computer Communication Review 84 Volume 37, Number 3, July 2007

Related documents

UntilNoChild FINAL

UntilNoChild FINAL

—25 YEARS— UNTIL NO CHILD HAS AIDS A COMMITMENT TO WOMEN, CHILDREN, AND FAMILIES

More info »
CDIR 2018 07 27

CDIR 2018 07 27

S. Pub. 115-7 2017-2018 Official Congressional Directory 115th Congress Convened January 3, 2017 JOINT COMMITTEE ON PRINTING UNITED STATES CONGRESS UNITED STATES GOVERNMENT PUBLISHING OFFICE WASHINGTO...

More info »
Zeitgeist, The Movie  Companion Guide PDF

Zeitgeist, The Movie Companion Guide PDF

ZEITGEIST: THE MOVIE COMPANION SOURCE GUIDE This material is protected by the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.

More info »
E:\DPS J0URNAL 12 2\206 212.pmd

E:\DPS J0URNAL 12 2\206 212.pmd

OCTOBER 2009 DELHI PSYCHIA JOURNAL V ol. 12 No.2 Y TR Review Article Love and Mental Health Safara Maryam, M.S. Bhatia Department of Psychiatry, University College of Medical Sciences & G.T.B. Hospita...

More info »
Scott and White Health Plan

Scott and White Health Plan

Scott and White Health Plan .fehb.swhp.org www Customer Service 800-321-7947 2019 Health Maintenance Organization (Standard and Basic Options) A This plan's health coverage qualifies as minimum essent...

More info »
Personal Values and the Counseling Relationship

Personal Values and the Counseling Relationship

Frances Patterson, PhD, LADAC, MAC, BCPC, CCJAS, QSAP, QCS Footprints Consulting Services, LLC

More info »
MCLA working paper FOR DIGITAL single pgs 08

MCLA working paper FOR DIGITAL single pgs 08

Lisa Donovan Leveraging Maren Brown Change: Increasing Access to Arts Education in Rural Areas WORKING PAPER

More info »
inline supplementary material 1

inline supplementary material 1

Supplementary Material able S1 T v2 and the Corresponding ICD - 9 and ICD - 10 Diagnosis and Procedure Chronic Care Conditions (CCC) . Categories of Codes : Feudtner C, Feinstein JA, Zhong W, Hall M, ...

More info »