8/13/2019 10 Random Lessons

1/24

10 (Random) Lessons

on Paper WritingRodolfo Pellizzoni

8/13/2019 10 Random Lessons

2/24

Introduction

Unfortunately, I only archive the final version of each paper, so Ican not show you a step -by-step paper analysis.

Luckily, I have been writing research papers for 5 years now, and

I had my fair share of ups and downs.

This is a collection of 10 (random?) lessons that I picked up andalways keep in mind.

Take them with a grain of salt, I am more interested in startingsome discussion than proving facts.

8/13/2019 10 Random Lessons

3/24

Lesson 1: Be sure to pick the rightconference / journal

It is very important to submit to a conference / journal whosetopic and interests closely match the contribution of your paper.

Even within the same community, different conferences canhave different focus!

For example, ECRTS is more oriented to theory, RTAS is more oriented tosystems. This also reflects in some way differences between thecommunity in Europe and the US.

Whenever possible, try to adjust the focus of the paper based

on the conference. Even just a slight reword of abstract and introduction can make a big

difference (see Lesson 3). Be careful with special tracks: acceptance ratios are typically not

higher than main track, submit only if the paper is very focused.

8/13/2019 10 Random Lessons

4/24

8/13/2019 10 Random Lessons

5/24

Lesson 2: Always check theprogram committee

Reviewers are people. They all have their different opinionsand points of view.

Whenever you want to submit to a conference, be sure tocheck the program committee first and identify possiblereviewers!

Things you should be careful about: Related work. Position on controversial issues (see Lesson 6). What they might be interested in seeing in the paper (see Lesson 8). Etc.

8/13/2019 10 Random Lessons

6/24

Lesson 3: Section 1 isthe most important

The introduction is by far the most important section in the entirepaper, especially for conferences.

Reviewers are always very busy. Grad students do a lot of reviews and the return on investments in

reviewing is poor. For conference papers, the author can not answer back so reviewers are

less accountable. If a reviewer can reject your paper without reading it all, it saves

time! The introduction is the first section they read, so make sure your

paper does not get killed in Section 1. 5 years ago I used to write the introduction last. Now it is always

the first section I write.

8/13/2019 10 Random Lessons

7/24

Example Every single paper of mine received comments that could have

been avoided with a better introduction. Instead of individual examples, here are some general suggestions:

1. Explain why the problem you solve is relevant and important. By far the most important point. Also the hardest to properly articulate.

Sometimes examples help, sometimes not.

2. Clearly state your contribution and why it is original. No original contribution - > incremental paper -> autorejection.

3. Summarize your main assumptions and limitations. People are skeptic that you found the holy grail.

4. Try to add something to encourage the reviewer to read on. This is an art that comes with experience.

8/13/2019 10 Random Lessons

8/24

Lesson 4: Intuitions are moreimportant than proofs

I consider myself a 50/50 person. That means I do 50% theory and50% implementation.

It also means I wrote papers that are full of theorems and lemmas.

Proofs are a necessary evils. We are a community of engineers not mathematicians, so reviewers will

not like a paper just because proofs are elegant. Theorem: reviewers do not like to read proofs.

Following Lesson 3, reviewers are busy. Since furthermore reading proofstakes a lot of time, the theorem follows.

Proofs also take a lot of space.

8/13/2019 10 Random Lessons

9/24

Lesson 4: Intuitions are moreimportant than proofs

Solution: always, always provide a good intuition for your resultbefore you show the proof.

If the reviewer is happy with the intuition, your paper will probably be ok(but see also Lesson 5).

I always review a submission without reading any proof first. If I paper islacking in contribution, there is no further need to read detailed proof.

You can even move all/some proofs to a technical report if you

do not have the space. There is no choice between putting the intuition and putting the detailed

proof in the paper. Intuition always win.

8/13/2019 10 Random Lessons

10/24

Example

R. Pellizzoni and M. Caccamo, Impact of Peripheral-ProcessorInterference on WCET Analysis of Real-Time Embedded Systems,Submitted to Transactions on Computers (Major Review).

Journal version of a previous RTSS conference paper.

Both papers are full of proofs. Conference reviewers complained about the complexity of

Theorem 4, which proves our main algorithm is correct. A rather complex proof by induction on a non-intuitive index.

The real problem is that we did not provide a good enoughintuition for the main algorithm. No journal reviewer complained about Theorem 4, even if the

proof is exactly the same.

8/13/2019 10 Random Lessons

11/24

Lesson 5: but make sure proofsare correct

Intuition is more important, but make sure all your proofs areformally correct!

If a conference reviewer finds any error, your paper is on afast track to rejection.

In journal submissions you usually get a chance to correct the

mistake if the result seems reasonable, but do not take that asan excuse to be careless.

8/13/2019 10 Random Lessons

12/24

8/13/2019 10 Random Lessons

13/24

Example3. The simpler the math, the better it is.

If you use anything more than standard algebra and calculus, be sure toprovide relevant links. Do not assume that everybody knows X justbecause every undergrad must study X in your department.

Some proof schemes tend to annoy people more than others.

8/13/2019 10 Random Lessons

14/24

Lesson 6: Strong statementsare dangerous

Be very careful when you make strong statements about someresearch issue: there are people that think otherwise.

Be especially careful when taking position on some hotlydebated topics in the community, like:

RM vs EDF. P-fair vs multiprocessor EDF. Partitioned vs global multiprocessor scheduling. Hard real-time wireless. Testing vs static analysis. Etc. etc. etc.

Instead of saying X is black, say X is usually black, but in somecases that are not considered in this paper it is white.

8/13/2019 10 Random Lessons

15/24

Example R. Pellizzoni and M. Caccamo: M-CASH: A Real-Time Resource

Reclaiming Algorithm for Multiprocessor Platforms. Real-TimeSystems Journal, 2008, Vol. 40(1): 117-147.

We said that we use multiprocessor EDF because while P-fairis optimal, it leads to excessive number of preemptions.

Reviewer commented: Some of the statements made aboutthe effects of preemptions are too strong: in fact, there existreal-time systems that use cached data rather seldomly(because new data is continually being processed).

8/13/2019 10 Random Lessons

16/24

Lesson 7: but if you areconfident, go for it!

However, high impact papers are those that successfullychallenge existing preconceptions.

So do not be shy when you state the main contribution ofyour paper!

If it is somehow controversial, you might have some troubles gettingthe paper accepted at first, but it is well worth in term of impact.

If it is not, you should still stress your contribution so the reviewer gets

more interested in the paper (remember Lesson 3)!

Just be sure to prove your point well enough; the keywordhere is successfully challenge.

8/13/2019 10 Random Lessons

17/24

Example R. Pellizzoni et al.: Coscheduling of CPU and I/O Transactions

in COTS-based Embedded Systems. RTSS 2008. The analysis in the paper computes a pattern of arrival times

for cache misses that causes max task WCET increase. I was scared because the analysis is built on top of my RTSS 07 paper

and furthermore the obtained WCET algorithm is pretty similar. In the introduction, I made the rather strong claim that the

result is very counterintuitive because the worst-case pattern

is the opposite of the classic real-time critical instant (cachesare spread out instead of arriving altogether).

Two reviewers out of four commented that the analysis is veryinteresting and new.

8/13/2019 10 Random Lessons

18/24

Lesson 8: You can not makeeverybody happy

Different reviewers are looking for different things (rememberLesson 2).

You must accept that it is simply impossible to make allreviewers perfectly happy; as shown in previous lectures, youare forced to make trade-offs.

For the same reason, take all reviewers comments with a grain of salt. The key: two half glasses of water are better than one full and

one empty glass here. Just one negative review is enough to kill a conference paper. Once your paper is out, if the contribution is solid people will start

citing you anyway.

8/13/2019 10 Random Lessons

19/24

Example Pellizzoni and P. Meredith and M. Caccamo and G. Rosu:

Hardware Runtime Monitoring for Dependable COTS-basedReal-Time Embedded Systems. RTSS 2008.

Paper is composed by three sections: Description of the main idea and monitoring theory (main part). Test case (very important to show that this new idea works). FPGA implementation (short because it is not the core of the paper,

but required to show that we implemented it for real).

Submitted to RTSS to a new design & verification track. Paper just barely got in.

8/13/2019 10 Random Lessons

20/24

Example Two main issues among reviewers.

Reviewers 1 & 2 commented that the FPGA implementationdetails were boring, and we should have expanded the formalmethod part (PTLTL monitor synthesis).

Reviewer 3 wrote: the paper sets out to describe ahardware solution. Yet there is very little actual hardwaredesign details.

8/13/2019 10 Random Lessons

21/24

Lesson 9: Retry, youllhave better luck

What you should get out of the previous lessons: you can do alot to improve your chances of being accepted, but you alsoneed some luck.

If you believe that your contribution is important, you shouldnot get discourage if your paper is rejected!

Out of my 12 original papers, 7 got rejected / major reviewed first! I managed to publish all 7 of them.

If you do not believe that your contribution is important, thanyou should not have written the paper in the first place!

8/13/2019 10 Random Lessons

22/24

Lesson 10: but be carefulwhere you resubmit!

Be careful if you submit twice in a row to a conference in the samearea, especially if the community is small like in real-time research.

Reviewers are people, and people are affected by negative biases.

If you want to resubmit, be sure to: Address all reviewers comment to the best of your ability, so the same

reviewer can not reject you twice for the same reason. Changing title/abstract/introduction can also help.

Otherwise, just can go directly for journal.

8/13/2019 10 Random Lessons

23/24

8/13/2019 10 Random Lessons

24/24

Lesson 0 It all boils down to the following Prime Directive:

Write for the reviewers,not for yourself!!!