Why bids fail: Bidding for EU ICT research projects

transcript

sbrown@dmu.ac.uk

http://kmd.dmu.ac.uk

Stephen Brown, 20 May 2010

sbrown@dmu.ac.uk

Overview

• FP7 ICT call 1 results• Project types• The evaluation process• Evaluation criteria• Scoring• Good proposals• Bad proposals

sbrown@dmu.ac.uk

Most proposals fail

sbrown@dmu.ac.uk

FP7 ICT call1 results

• 188 eligible proposals submitted

• 12 proposals funded (52m Euros)

• 6% success

Most proposals fail…..

……but not because they are bad

sbrown@dmu.ac.uk

Project type Funding requested

Funding published

IPs 181.5 M€ 20-32.5 M€

STREPs 506.8 M€ 10-22 M€

NoEs 15.6 M€ 5 M€

CSAs 14.2 M€ 2.5 M€

Most proposals have to fail

sbrown@dmu.ac.uk

Project types

• STREPs

• IPs

• NoEs

• CAs

• SAs

sbrown@dmu.ac.uk

STREPs

• Focused objectives• Clearly identified problem• Research and demonstration activities• Scope for competing approaches to solving

problems• Small scale (2-4 M€ over 1-2 years)

sbrown@dmu.ac.uk

• Integration of projects within a coherent set of activities

• Outreach and validation are important• Include training, innovation, takeup and

dissemination activities• Active partners with substantial roles and clear

responsibilities• Large IPs (average range is 6-9 M€ over 3-4

years) need to show very clearly how they will make a significant impact in their target area

sbrown@dmu.ac.uk

• Aim is the durable integration of high calibre research capacity

• NoEs should involve the stakeholders, especially industry

• Funding is for convergence and embedding, not research

• Size and funding of NoEs much smaller than in FP6 (2-3M€ over 3 years)

sbrown@dmu.ac.uk

• Bringing researchers together either in new areas (as forerunner of eventual NoEs)

• Supporting workshops and communities of practice – eg in creating framework conditions for take up of research work

• Support for building and maintaining the body of evidence of research

• <1M € each

sbrown@dmu.ac.uk

The evaluation process

sbrown@dmu.ac.uk

The evaluation process

STREPs• Individual expert• Consensus Group• Panel meeting

IPs & NoEs• Individual expert• Consensus Group• Interim panel meeting• Hearing• Panel meeting

Plenty of opportunities to fail

sbrown@dmu.ac.uk

Evaluation criteria

sbrown@dmu.ac.uk

S/T QUALITY“Scientific and/or

technological excellence

(relevant to the topics addressed by the call)”

IMPLEMENTATION“Quality and efficiency of the

implementation and the

management”

IMPACT“Potential impact through the

development, dissemination

and use of project results”

• Soundness of concept, and quality of objectives

• Progress beyond the state-of-the-art

• Quality and effectiveness of the S/T methodology and associated work plan

• Appropriateness of the management structure and procedures• Quality of the consortium as a whole (including complementarity, balance)• Quality and relevant experience of the individual participants• Appropriateness of the allocation and justification of the resources to be committed (budget, staff, equipment)

• Contribution, at the European and / or international level, to the expected impacts listed in the work programme under relevant topic/activity• Appropriateness of measures for the dissemination and/or exploitation of project results, and management of intellectual property

sbrown@dmu.ac.uk

Scoring0 - The proposal fails to address the criterion under examination or

cannot be judged due to missing or incomplete information.1 - Very poor. The criterion is addressed in a cursory and

unsatisfactory manner.2 - Poor. Serious inherent weaknesses in relation to the criterion in

question.3 - Fair. While the proposal broadly addresses the criterion, there aresignificant weaknesses that would need correcting.4 - Good. The proposal addresses the criterion well, although certainimprovements are possible.5 - Excellent. The proposal successfully addresses all relevant

aspects of the criterion in question. Any shortcomings are minor.

sbrown@dmu.ac.uk

Funding thresholds

• Proposals must score at least 3 on any criterion to be funded

• Proposals scoring 10 or above can be considered for funding

• 35% above threshold but not funded

• Proposals scoring 13.5 or above are usually considered for funding

• You need drop only 3 half marks to fail

sbrown@dmu.ac.uk

• 67% failed on multiple thresholds

• of these 43% failed on criterion 1 – scientific excellence relevant to the objectives

• 52% IPs and 40% STREPS failed criterion 3 - impact

sbrown@dmu.ac.uk

Don’t give up

sbrown@dmu.ac.uk

What good proposals look like

• Describe your problem, explain why it is relevant and how you will tackle it

• Describe the specific state-of-the-art with referenced evidence, as well as the technical baseline, and expected advancements against which progress can be measured

• Show you understand the state of the art – don’t just list projects and articles

• Check the timelines and anticipated outputs of ongoing research in defining your starting point and the advances you will make – don’t replicate existing work

sbrown@dmu.ac.uk

What good proposals look like

• Explain how you will ensure impact • Adopt a scientifically sound approach to

involving users in the research, including to the assessment and validation necessary to build the evidence of impact

• Find the right partners – not necessarily the nearest or most convenient. Do justice to the multi-disciplinary nature of the area – ensure the expertise and the roles are balanced and appropriate.

• Cost out work packages clearly and realistically

sbrown@dmu.ac.uk

Relevance

sbrown@dmu.ac.uk

Work Programme

• Target outcomes

• Fifth call results

sbrown@dmu.ac.uk

Common mistakes

sbrown@dmu.ac.uk

• Provide yet another training solution for a particular set of users (eg training for engineers) with no new work on how people acquire skills and competences, in different contexts

• Develop a Learning Management System, Content Delivery Platform or VLE – these are mainstream eLearning products

• Develop something for a specific language, geography, history and don’t justify how ICTs will improve learning in that field

sbrown@dmu.ac.uk

• Describe a “technology driven” type of project• Fail to leverage a balance of research across

the contributing disciplines• Fail to identify what the different disciplines

contribute• Produce a proposal that tries to do everything

and is just not credible. Often less is more

sbrown@dmu.ac.uk

Critical questions youshould ask yourselves

• Does the proposal address the right published target outcomes?

• Does the proposal address a new problem or offer different and innovative insights into an existing one?

• How far is the problem you intend to address already being tackled elsewhere?

• Which communities are likely to benefit from the project / how are they involved?

• What will the benefits / impact of the project be?• What are the challenges and potential risks and how are

they tackled?

sbrown@dmu.ac.uk

Further information

• Comprehensive EC guide to “How to fail”– ftp://ftp.cordis.europa.eu/pub/ist/docs/telearn/what-

not-to-do_en.pdf

• Introduction to Cultural Heritage & Technology Enhanced Learning, including links to programme descriptions, publications that describe currently funded projects and links to commission / project web sites and contacts.– http://cordis.europa.eu/fp7/ict/telearn-digicult/

home_en.html

sbrown@dmu.ac.uk

Stephen Brown

For further information contact

Why bids fail: Bidding for EU ICT research projects

Education