Oficina de Suport a la Recerca i la
Transferència de la UOC
Cicle Dijous amb l’OSRT
European Research Council
(ERC) a l’Horizon 2020
Teresa Puig,
Institut de Ciència de Materials de Barcelona
Barcelona, 30 de gener de 2014
http://w.uoc.edu/osrt
ERC Evaluator Perspective
Teresa Puig
Insitut de Ciència de Materials de Barcelona
CSIC
Bellaterra, Spain
January 2014
Who am I ?
Evaluator for St and Co grants in 2012 in PE3
(Condensed Matter Physics). Two streams
One of the over 375 evaluators from the 25
panels (3 domain): ∼ 5000 proposals
Recruited by the ERC Scientific Council
Assigned to one panel according to CV and
expertise
% of confidence in 20 specific fields (descriptors)
within the area of the panel
Identify 10 free keywords defining my expertise
Establish my current interest
PhD in Physics (20 month in Sweden, Ireland, Germany)
Postdoctoral research in Belgium
Head of the Department of Superconducting Materials and large
scale nanostructures at ICMAB (25 researchers, SGR2009-770)
Editorial Executive Board of SUST, Board of ESAS
Co-founder of OXOLUTIA S.L. , spin-off from ICMAB
PI of several National and EU projects
European projects (NMP, Energy, ITN, COST) in the field of
superconducting materials and their integration in power devices
Duran Farrell- Gas Natural, Novare-Endesa awards
200 peer review papers, 9 patents , 13 PhD Thesis, 30 invited talks
Evaluator of AGAUR, MICINN, ANEP, MINCyt-Argentina, ERCEA
My CV
ERC comments to Evaluators
ERC funds frontier research in Europe
Excellent Ideas
All fields of science and humanities without
thematic priorities
Individual scientists
No quotas, neither on PI nationality, HI nationality,
gender
One selection criteria: EXCELLENCE
High risk/ High gain
Ground-breaking ideas and not good excellent
research
Evaluation Panel
Each panel is composed of 12-15 panel members (PM)
One acts as Panel Chair (PC)
1 Panel Coordinator from Scientific Council
Good balance between different fields of panel area
Panel composition has maximum two members from
one country
PM composition is decided half a year before
proposals submission. PM changes in consecutive years
Conflict of interest (CoI) strictly attained for each
individual proposal
Non-PM can submit a proposal at the same call
Remote external reviewers
Typically 2000/call
Evaluate a small number of proposals
Answer the same evaluation questionnaire as PM
Crucial in the second stage evaluation
Recruited by PM upon acceptance of Scientific
Council
Referees
Evaluators Process steps
Two step process : Right balance between
generalist and specialized reviewers
Stage 1
B- Remote assessments by panel members
(and external referees if assigned)
PI and Synopsis evaluation only
C- Panel meeting
Decision of proposals retained for step 2
D- Feedback to applicants: Grade A, B, C ( only A is retained )
A- PC assigns proposals to panel members (3-4/RP) (evaluations
could be requested to other panels). Each PM evaluates 40-50 prop.
F- Remote assessments by PM and referees (6-7/RP)
Full proposal evaluation includes budget
Each PM evaluates ∼ 10 proposals.
Also evaluations from external referees
G - Panel meeting + interview
Ranked list of proposals
Stage 2
H- Feedback to applicants: Decision for proposals to be funded.
All proposals receive panel comments
I- Redress cases
E- Assign external referees to retained proposals
Evaluators Process steps
Evaluation Criteria
Published in the call
Read very carefully at different stages of the
writing process before submission
Excellence of PI:
Intellectual capacity
Creativity
Commitment
Excellence of RP (research project)
Ground breaking nature
Potential impact
Scientific approach
Each criteria is ranked
Select the right Panel. It can be reassigned
Carefully ensure you considered all evaluation criteria
Guarantee you followed templates indications (use
criteria titles/subtitles) including length
Choose carefully your descriptors and free keywords
best defining your proposal
They are extremely important for reviewers assignment
Matching with those from reviewers
You want the best reviewers for your proposal
The proposal must be outstanding not excellent
(evaluation criteria 1-4, 3= excellent, 4= outstanding)
Use figures , charts . Right references are crucial.
Proposal submission
Recommendations
Stage 1 Evaluation : Recommendations
Have a well presented CV. As important as the project
Fulfill all requested information. Clearly identify your PhD
supervisor
Researcher ID and Group-Web address are best given
Most reviewers will like to check
Demonstrate independent creative thinking with past
publications. Explain your transition to scientific independence
PI evaluation
Intellectual capacity and creativity
Remote evaluation
Part B1 (1)
Be concise, understandable, appealing for generalists
and expert reviewers
Clarify context, clearly identify the problem to be
solved, what the gain is, why it should be funded
Demonstrate ground-breaking nature of the RP
Convince that you address an important challenge at
the knowledge frontier
Ambitious objectives well beyond the state of the art
Specify if novel concepts/approaches
RP synopsis evaluation
Ground breaking nature and potential impact of the research
Stage 1 Evaluation : Recommendations
Remote evaluation
Part B1 (2)
Appropriate selection of methodology to reach the
goals
Specify if need for a novel and/or unconventional
methodology. It depends on the RP
Justify high risk/high gain balance. Identify the risks,
some contingency plan might be good
Proposal needs to be feasible. Do not over-dimension the
work plan
Proposal abstract/summary is very important. First
read by the reviewer. Take your time writing it
RP synopsis evaluation
Methodology
Stage 1 Evaluation : Recommendations
Remote evaluation
Part B1 (3)
Proposals are ranked in a list according to remote
evaluation
Discussed one by one
Usually, ∼20% ranked A (retained), ∼50% ranked B,
30% ranked C
Discrepancies among PM usually are in top B /
bottom A proposals
All B and C- proposals receive the PM comments
(discussion) together with all remote evaluations
(unchanged). Read carefully PM comments
A- proposals are requested for interview in stage 2
Panel meeting and retained-proposals decision
Stage 1 Evaluation
PI and RP (Parts B1 and B2) being again evaluated
New referees (experts) come into the evaluation
Same evaluation criteria as 1st stage, but now with
full project proposal
New things:
How well conceived and organized is the activity
Demonstrate that the goals of the proposal can be
achieved with timescale and resources available
Describe accurately the requested budget vs. RP
Full proposal evaluation
Stage 2 Evaluation : Recommendations
Remote evaluation
Part B2
Proposals are ranked in a list according to remote
evaluation. Discussed one by one
All PM evaluate all interviews (except CoI)
Proposal’s lead reviewer (PM) directs the interview
Questions by external referees are raised at interview
Give a copy of presentation to all PM (∼15)
It helps to remember PI/RP during the final discussion
Bring extra slides for possible questions
If new preliminary results, show them (it’s 6 month later)
Be aware of recent publications of the field
Panel discussion already starts after your interview
It has several stages
Panel meeting and interview
Stage 2 Evaluation : Recommendations
Interview evaluation
Rehearse many times (10’ talk +15’ questions)
Demonstrate your capacity, be convinced of your RP
Talk in 1st singular. Demonstrate maturity. You deserve it now, not
next year. Be enthusiastic !.
Short presentation: Excellent Idea is most important
Do not start explaining your CV. Key evaluators know it.
Demonstrate importance of your past publications linked to the RP
Few slides, be concise and clear, no need of details
Generalist won’t follow and experts know them from the proposal
Go straight to the point: What the problem to be solved is,
how you will solve it
Answer concisely, precisely, allow for many questions
Make the full panel be interested in your proposal
Stage 2 Evaluation : Recommendations
Final decision and feedback to applicants
Decisions are taken by all the panel
All proposals need to be ranked in one single list
Outstanding proposals are usually agreed by most of
PM and not-to-be funded proposal too. Discrepancies
come at the “grey list”
Your impression at the interview is a key factor
Budget is not an elimination criterion. If not properly
justified, it will be cut down
Usually, ∼10-12% from overall proposals are funded
All proposals receive the PM comments
(discussion)together with all remote evaluations
(unchanged). Read carefully PM comments
Stage 2 Evaluation
“Write the best proposal you can
imagine and make it outstanding,
understandable for a generalist of
your field and appealing for experts”
“You’ll only win if you participate”
Good luck

European Research Council a l'Horizon 2020

  • 1.
    Oficina de Suporta la Recerca i la Transferència de la UOC Cicle Dijous amb l’OSRT European Research Council (ERC) a l’Horizon 2020 Teresa Puig, Institut de Ciència de Materials de Barcelona Barcelona, 30 de gener de 2014 http://w.uoc.edu/osrt
  • 2.
    ERC Evaluator Perspective TeresaPuig Insitut de Ciència de Materials de Barcelona CSIC Bellaterra, Spain January 2014
  • 3.
    Who am I? Evaluator for St and Co grants in 2012 in PE3 (Condensed Matter Physics). Two streams One of the over 375 evaluators from the 25 panels (3 domain): ∼ 5000 proposals Recruited by the ERC Scientific Council Assigned to one panel according to CV and expertise % of confidence in 20 specific fields (descriptors) within the area of the panel Identify 10 free keywords defining my expertise Establish my current interest
  • 4.
    PhD in Physics(20 month in Sweden, Ireland, Germany) Postdoctoral research in Belgium Head of the Department of Superconducting Materials and large scale nanostructures at ICMAB (25 researchers, SGR2009-770) Editorial Executive Board of SUST, Board of ESAS Co-founder of OXOLUTIA S.L. , spin-off from ICMAB PI of several National and EU projects European projects (NMP, Energy, ITN, COST) in the field of superconducting materials and their integration in power devices Duran Farrell- Gas Natural, Novare-Endesa awards 200 peer review papers, 9 patents , 13 PhD Thesis, 30 invited talks Evaluator of AGAUR, MICINN, ANEP, MINCyt-Argentina, ERCEA My CV
  • 5.
    ERC comments toEvaluators ERC funds frontier research in Europe Excellent Ideas All fields of science and humanities without thematic priorities Individual scientists No quotas, neither on PI nationality, HI nationality, gender One selection criteria: EXCELLENCE High risk/ High gain Ground-breaking ideas and not good excellent research
  • 6.
    Evaluation Panel Each panelis composed of 12-15 panel members (PM) One acts as Panel Chair (PC) 1 Panel Coordinator from Scientific Council Good balance between different fields of panel area Panel composition has maximum two members from one country PM composition is decided half a year before proposals submission. PM changes in consecutive years Conflict of interest (CoI) strictly attained for each individual proposal Non-PM can submit a proposal at the same call
  • 7.
    Remote external reviewers Typically2000/call Evaluate a small number of proposals Answer the same evaluation questionnaire as PM Crucial in the second stage evaluation Recruited by PM upon acceptance of Scientific Council Referees
  • 8.
    Evaluators Process steps Twostep process : Right balance between generalist and specialized reviewers Stage 1 B- Remote assessments by panel members (and external referees if assigned) PI and Synopsis evaluation only C- Panel meeting Decision of proposals retained for step 2 D- Feedback to applicants: Grade A, B, C ( only A is retained ) A- PC assigns proposals to panel members (3-4/RP) (evaluations could be requested to other panels). Each PM evaluates 40-50 prop.
  • 9.
    F- Remote assessmentsby PM and referees (6-7/RP) Full proposal evaluation includes budget Each PM evaluates ∼ 10 proposals. Also evaluations from external referees G - Panel meeting + interview Ranked list of proposals Stage 2 H- Feedback to applicants: Decision for proposals to be funded. All proposals receive panel comments I- Redress cases E- Assign external referees to retained proposals Evaluators Process steps
  • 10.
    Evaluation Criteria Published inthe call Read very carefully at different stages of the writing process before submission Excellence of PI: Intellectual capacity Creativity Commitment Excellence of RP (research project) Ground breaking nature Potential impact Scientific approach Each criteria is ranked
  • 11.
    Select the rightPanel. It can be reassigned Carefully ensure you considered all evaluation criteria Guarantee you followed templates indications (use criteria titles/subtitles) including length Choose carefully your descriptors and free keywords best defining your proposal They are extremely important for reviewers assignment Matching with those from reviewers You want the best reviewers for your proposal The proposal must be outstanding not excellent (evaluation criteria 1-4, 3= excellent, 4= outstanding) Use figures , charts . Right references are crucial. Proposal submission Recommendations
  • 12.
    Stage 1 Evaluation: Recommendations Have a well presented CV. As important as the project Fulfill all requested information. Clearly identify your PhD supervisor Researcher ID and Group-Web address are best given Most reviewers will like to check Demonstrate independent creative thinking with past publications. Explain your transition to scientific independence PI evaluation Intellectual capacity and creativity Remote evaluation Part B1 (1)
  • 13.
    Be concise, understandable,appealing for generalists and expert reviewers Clarify context, clearly identify the problem to be solved, what the gain is, why it should be funded Demonstrate ground-breaking nature of the RP Convince that you address an important challenge at the knowledge frontier Ambitious objectives well beyond the state of the art Specify if novel concepts/approaches RP synopsis evaluation Ground breaking nature and potential impact of the research Stage 1 Evaluation : Recommendations Remote evaluation Part B1 (2)
  • 14.
    Appropriate selection ofmethodology to reach the goals Specify if need for a novel and/or unconventional methodology. It depends on the RP Justify high risk/high gain balance. Identify the risks, some contingency plan might be good Proposal needs to be feasible. Do not over-dimension the work plan Proposal abstract/summary is very important. First read by the reviewer. Take your time writing it RP synopsis evaluation Methodology Stage 1 Evaluation : Recommendations Remote evaluation Part B1 (3)
  • 15.
    Proposals are rankedin a list according to remote evaluation Discussed one by one Usually, ∼20% ranked A (retained), ∼50% ranked B, 30% ranked C Discrepancies among PM usually are in top B / bottom A proposals All B and C- proposals receive the PM comments (discussion) together with all remote evaluations (unchanged). Read carefully PM comments A- proposals are requested for interview in stage 2 Panel meeting and retained-proposals decision Stage 1 Evaluation
  • 16.
    PI and RP(Parts B1 and B2) being again evaluated New referees (experts) come into the evaluation Same evaluation criteria as 1st stage, but now with full project proposal New things: How well conceived and organized is the activity Demonstrate that the goals of the proposal can be achieved with timescale and resources available Describe accurately the requested budget vs. RP Full proposal evaluation Stage 2 Evaluation : Recommendations Remote evaluation Part B2
  • 17.
    Proposals are rankedin a list according to remote evaluation. Discussed one by one All PM evaluate all interviews (except CoI) Proposal’s lead reviewer (PM) directs the interview Questions by external referees are raised at interview Give a copy of presentation to all PM (∼15) It helps to remember PI/RP during the final discussion Bring extra slides for possible questions If new preliminary results, show them (it’s 6 month later) Be aware of recent publications of the field Panel discussion already starts after your interview It has several stages Panel meeting and interview Stage 2 Evaluation : Recommendations
  • 18.
    Interview evaluation Rehearse manytimes (10’ talk +15’ questions) Demonstrate your capacity, be convinced of your RP Talk in 1st singular. Demonstrate maturity. You deserve it now, not next year. Be enthusiastic !. Short presentation: Excellent Idea is most important Do not start explaining your CV. Key evaluators know it. Demonstrate importance of your past publications linked to the RP Few slides, be concise and clear, no need of details Generalist won’t follow and experts know them from the proposal Go straight to the point: What the problem to be solved is, how you will solve it Answer concisely, precisely, allow for many questions Make the full panel be interested in your proposal Stage 2 Evaluation : Recommendations
  • 19.
    Final decision andfeedback to applicants Decisions are taken by all the panel All proposals need to be ranked in one single list Outstanding proposals are usually agreed by most of PM and not-to-be funded proposal too. Discrepancies come at the “grey list” Your impression at the interview is a key factor Budget is not an elimination criterion. If not properly justified, it will be cut down Usually, ∼10-12% from overall proposals are funded All proposals receive the PM comments (discussion)together with all remote evaluations (unchanged). Read carefully PM comments Stage 2 Evaluation
  • 20.
    “Write the bestproposal you can imagine and make it outstanding, understandable for a generalist of your field and appealing for experts” “You’ll only win if you participate” Good luck