How to conduct a Delphi Consensus Process
A Delphi consensus process can be employed to generate expert opinion evidence on a particular topic. It involves sending out a series of questionnaires to gather anonymous expert feedback, each round building on the previous one.
Nowadays, many industries including healthcare use the Delphi method to better understand situations, forecast, or inform decision-making and policy.
There is not a single agreed-upon methodology for the Delphi consensus process, in fact, most Delphi studies employ ‘modified’ Delphi methods, that are specifically designed based on project requirements and intended use of data. That said, studies using the Delphi consensus process must remain within a certain framework and adhere to key principles to ensure the validity of results. This article outlines the key phases of a modified Delphi consensus process and which steps they entail.
Scoping the study and engaging the experts
A Delphi consensus process kicks off with a clear definition of the aim and objectives. Designing the study to answer a specific research question can be beneficial although not a prerequisite. Conducting a literature or context review can also be highly valuable. It grounds the research in existing evidence and informs the direction of the Delphi consensus process, for instance, to reinforce, add to, or contradict existing data.
As a successful Delphi consensus process relies on strong project management, the scoping phase should also encompass planning the project’s logistics. This requires assessing which resources will be needed including stakeholders’ involvement and research costs, as well as evaluating the timeframe for the study.
In a Delphi consensus process, identifying and recruiting the appropriate experts is vital to ensure the validity of the study’s outcomes. Experts in the panel could range from 30 to several hundred and must demonstrate knowledge and ability, usually through criteria like specific qualifications, years of experience, or publications in the research domain. It is also key for the panel to be representative, to get diverse perspectives and comprehensive insights. This might mean including experts from different roles, geographical areas, or demographics depending on the project’s scope.
Designing the questionnaire, gathering and analysing the data
In a Delphi consensus process, the first questionnaire to be sent out to responders can be developed by asking open-ended questions to some experts in the field or by relying on literature review or steering group input to generate statements. The decision might depend on the availability of existing knowledge, on how much relevant knowledge experts carry on the question despite that knowledge not being properly established yet, as well as logistics. The questionnaire can then be disseminated to the panel of experts to survey their level of agreement with statements, often using a Likert scale and free-text to understand reasons for any disagreement in particular.
Once the first questionnaire has been shared, an iterative process starts during which the questionnaire is reviewed and modified until a satisfactory level of agreement is found, usually the threshold set around 70%-75%. The researcher then conducts a quantitative analysis of the results, although a qualitative analysis can be used to add more context to the results, for instance if comments from experts were gathered for further consideration.
The key principle to adhere to at this stage is anonymity of response – each panel member should be unaware of any other individual panellists responses. This protects against undue bias of results.
Closing the study and disseminating the results
Once data has been analysed, it is important to come back to the Delphi consensus process’ objectives that had been set at the start. Those will inform to who and how results should be communicated. Deriving recommendations from the data and including them in the manuscript can be a way to make findings more practical to implement and helpful to healthcare professionals for instance. The perceived credibility of a study, particularly in certain industries like healthcare, often hinges on publication so submitting for peer review is advised but not always necessary.
However, additional efforts are usually required to ensure the new evidence generated is as accessible as possible. A campaign to disseminate findings beyond publication might prove necessary to facilitate awareness of the work and to support tangible real-world changes. Different audiences within healthcare may require a different strategy, for instance, the production of targeted materials such as scientific presentations for the academic community, policy tools for payers, or educational brochures for patients. Effective dissemination of knowledge is what will influence behavioural changes, guidelines, and treatment pathways, leading to real-world healthcare improvements.
Running a successful Delphi consensus process requires both knowledge and hands-on experience. For a study to be credible and ultimately hold the power to inform decisions and bring change, a clear methodology that respects the key principles of the Delphi method must be followed.
About us
At Triducive we deliver consensus-led evidence that gets published and supports change. Our team, with a strong background in healthcare, has been delivering Delphi consensus for over ten years, boasting over 50 peer reviewed publications, and fostering positive change.
Get in touch with us to learn about our work with the modified Delphi consensus process.