International Early Stage Researcher Training School on Applying Expert  
Judgement Methodologies to Real Problems  

ICMAT, Campus de Cantoblanco, Madrid   

April 12 - 15, 2015   

Instituto de Ciencias Matemáticas
C/ Nicolás Cabrera, 13-15
  Campus Cantoblanco - UAM
  28049 Madrid, Spain
  Telephone: +34 91 2999700


Outline Programme

Sunday 12th April 2015 @Restaurant (to be confirmed) in Madrid

Arrival by late afternoon in Madrid.


Welcome Reception at a restaurant, including introductions, a brief overview of the course, a short elicitation exercise and Dinner.

Led by Simon French.

Car transit for those staying at Erasmo residence.



Monday 13th April 2015 @ICMAT-CSIC, Nicolás Cabrera 13-15, 28049 Madrid

08:15: Bus transit to ICMAT-CSIC.
09:00 Welcome and Introduction to ICMAT-CSIC and its facilities.

David Ríos.


Lecture on Cooke’s Classical Method, including some case study material, and demonstration of the Excalibur software.

Roger Cooke.


Lecture on the behavioural background: System 1 and System 2 thinking, heuristics and biases, group behaviours, relevance to debates about the relative merits of behavioural and mathematical aggregation.

Eva Chen.


Coffee and small group discussion with mentors. Note that the course directors will provide the mentors with a series of suggestions and topics for discussion to ensure that these sessions have structure.

Groups led by mentors.


Experiential Exercise: Feedback on the previous evening’s results. Training in how to make judgements, i.e. the participants will experience the training that they would need to provide as analysts to their experts. Structured elicitation of their individual judgements on further calibration variables and quantities of interest that will become resolved by Day 2.

Initial feedback. Simon French.
Trainer: Oswaldo Morales.
Panel: Oswaldo Morales (chair), Roger Cooke, Eva Chen, Simon French, Jesus. Rios, David Rios.


Short Presentation of a Case Study.

Jesus Rios. Strategic financial planning.

13:30 Lunch and free study time for students might consult materials on the Action’s Moodle VLE site.

Introduction to the group exercise in a plenary session. Overall description of the exercise. The participants will be divided into groups and given a topic on which several of the lecturers and mentors can act as experts – i.e. from experience of past applications they have sufficient contextual knowledge to provide judgemental inputs. Note: these groups need not necessarily be the same as the groups meeting at coffee time.

Simon French


Group work identifying key uncertainties on which they would need expert judgemental input in their exercise, potential calibration data sets, etc. Each group would be assigned a mentor/tutor to provide guidance on process, tools and methods, but not on context. Note each group will work on the same problem but independently.

Groups with mentors.

18:30 End of afternoon session.

Drinks and talk by guest speaker on their experience of the use of expert judgement in an applied study or studies.

Oswaldo Morales.

20:00 Buffet Dinner at ICMAT-CSIC and bus back to central Madrid.



Tuesday 14th April 2015@ICMAT-CSIC, Nicolás Cabrera 13-15, 28049 Madrid

08:15: Bus transit to ICMAT-CSIC.

Lecture on expert judgement theory (other than Cooke’s Method) covering topics such as Bayesian approaches, group decision making, meta-analysis etc.

Simon French


Short talk covering case studies or research fronts that the mentors/course lecturers are involved with.

David Rios. Aviation Safety Project.


Experiential Exercise: Session 3. Feedback and discussion of the results from the previous day exercises. Learning points, etc.

Simon French


Coffee and small group discussion with mentors.

Groups with mentors.


Design, reporting and peer review of expert judgement studies. Lecture and discussion. The aim of this session is to emphasise the softer professional skills needed in the community of expert judgement analysts to ensure that such studies are carried out rigorously and defensibly. The session will involve an exercise critiquing one or more reports of expert judgement studies. (Note: participants will have been given copies of these in their packs on arrival and asked to read them in their free study time).

Led by Roger Cooke.


Short talk covering case studies or research fronts that the mentors/course lecturers are involved with.

Eva Chen. The good judgement project.
Simon French and David Hartley. Bayesian approaches to calibration and combination.

13:30 Lunch and free study time allowing groups to work on their studies.

Group Exercise. Elicitation of judgements from ‘experts’. The mentors and other course lecturers will visit the groups in rotation, role playing experts for this. Analysis and development of a group report on the results.

Groups with mentors.

19:00 End of afternoon session. Bus back to centre.

School Dinner in Central Madrid.

Car transit for those staying at Erasmo residence



Wednesday 15th April 2015 @ICMAT-CSIC, Nicolás Cabrera 13-15, 28049 Madrid

08:15: Bus transit to ICMAT-CSIC.

Group exercise. Finalising analyses, reports and developing a presentation on the results of the exercise. (Coffee to be available to the groups).

Groups with mentors.


Group presentations of the results of the exercise and discussion.

Simon French (chair).

11:00 Coffee.

General discussion on how they will take their skills forward in their research and professional activity.

Roger Cooke (chair).

12:00 Close of course.






© ICMAT - 2008-2015, All Rights Reversed.