Expert elicitation processes are increasingly applied in environmental policy analyses, to deal with complex phenomena characterised by lack or scarcity of data. Judgments and information obtained from the experts are applied to overcome uncertainty limits of analytic modelling, to support decision-making processes, to populate knowledge and quantitative models and to enhance the synergism of inter-experts discussion.
In the ICARUS project, the choice to elicit and use experts’ judgments arises from a combination of the above reasons. Selected experts are involved for their deep knowledge of the innovative technologies under investigation. The purpose of the elicitation exercise is to collect quantitative data, probabilistic estimates and qualitative information on the technical characteristics of each technology, on the main barriers to their success and diffusion into the market, and on the role of RD&D investments in helping to overcome their limits and abate the costs.
While group interaction among experts is considered in the literature as the best method for gathering a large amount of qualitative data, the individual interview is suggested as the most suitable approach for obtaining detailed quantitative data, and also for analysing the expert’s problem-solving process [1]. Moreover, the face-to-face interview avoids potential biases from group dynamics which can break the link between subjective perception and underlying objective reality [2].
Unfortunately, the face-to-face elicitation situation is extremely expensive and time-consuming. As a consequence, it is often substituted with less effective techniques, such as the submission of questionnaires by email or by phone. These modes of communication do not allow the analyst to carefully monitor the whole exercise and to control the occurrence of biases and errors in the estimates [3].
The ICARUS project gave us the possibility to test an effective solution aimed at combining the advantages of a face-to-face interview, with the possibility to save money, time and carbon emissions. We carried out most of the interviews through web conferences, using a meeting software that can be accessed from everywhere. The expert is invited to join an online meeting, where we present information and share documents and applications. The expert has therefore the opportunity to compile the questionnaires on line, while discussing questions and commenting the survey. As she/he modifies the document, we can see changes and intervene in real time to clarify doubts or to ask for more specific information. All interviews can be taped not to lose the ancillary information, which can be used to check and contextualise the experts’ answers and, if necessary, to structure follow-up phone calls. The expert can see the interviewers through the webcam, and this helps to recreate the overall effect of a face-to-face situation. We have so far undertaken more than 30 web interviews and always received very positive feedbacks for the flexibility and professionalism of the procedure. The web meeting approach resulted effective and user-friendly, ensuring the defensibility of the elicitation outputs and the robustness of the consequent analyses, at the same time making a larger number of interviews and follow-ups possible.
References
- [1] Meyer, M.A., Booker, J.M., (1991), Eliciting and Analysing Expert Judgment: A Practical Guide. Academic Press Ldt., London.
- [2] Baddeley, M., Curtis, A., Wood, R., (2004), An introduction to prior information derived from probabilistic judgments: Elicitation of knowledge, cognitive bias and herding, In: Curtis, A., Wood, R., editors, Geological Prior Information, Geol. Soc. Lond. Special Publication 239: 1-14.
- [3] O’Hagan, A., Buck, C.E., Daneshkhan, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J., Oakey, J.E., Rakow, T., (2006), Uncertain Judgments: Eliciting Experts’ Probabilities, John Wiley & Sons, Ltd.
