5 Biggest Challenges for Delphi Consensus
Having delivered Delphi consensus for healthcare evidence-generation for the past 10 years, our team has learned to anticipate specific challenges that may arise during the process.
Here are 5 of the biggest challenges to overcome when leading a Delphi Consensus:
1. Defining the Objectives of a Delphi Consensus
Agreeing on the right objectives for the Delphi Consensus from the start can be challenging. If they are too broad, this will negatively affect the direction of the consensus study. It becomes a struggle to produce statements that are relevant and powerful. This will undermine the potential impact of the consensus. Ultimately, this can alter perceptions regarding the credibility and legitimacy of the project.
- What is wrong with the status quo/what gap exists?
- Why is this the case (evidence/guidelines/conventions etc.)?
- What do we think should happen instead….and why?
Answers to these questions will enable a much clearer focus for the study to emerge and this can be further refined by a targeted review of available published evidence.
Establishing a clear, shared understanding across the project team of the Delphi study aim will create alignment to achieve stronger results.
2. Managing the Project Timeline of a Delphi Consensus
Delphi studies typically span a number of months, and there is always a risk of projects dragging on longer than necessary due to a lack of organization or competing priorities. This not only hinders momentum and reduces team motivation but also delays the impact of the study, preventing change from happening sooner.
This is why effective project management is crucial. Leading a Delphi Consensus requires seamless coordination, managing communication internally and externally and ensuring the plan in place is followed. This plan should outline expectations regarding necessary input from stakeholders involved throughout the project. Providing a clear direction with regular updates helps maintain the willpower to progress efficiently. This also means that even if timelines slip, adjustments are usually adopted with greater urgency.
3. Defining Delphi Statements
The definition of statements is a critical phase of the Delphi consensus process and, when not executed accurately, has the potential to compromise the quality of the study. Success is dependent upon achieving consensus regarding the core statements that will act as the foundation for instigating change in healthcare. Statements that are worded in an overly broad way that can be interpreted with ambiguity will risk limiting agreement from experts, and are also impractical for implementation.
To provide robust real-world evidence from a Delphi study, statements must effectively address the core questions/gaps identified and provide potential for clear recommendations. They should be formulated using language that is impartial and objective, eliminating any vagueness. The intent is to ensure that the meaning of statements remains indisputable and open to no varied interpretations when used to drive subsequent actions.
Well-defined statements will enable the creation of valid real-world evidence from a Delphi study.
4. Determining Consensus for a Delphi Study
Determining consensus poses a challenge, given the unique nature of each healthcare innovation. Without a clear analytical framework in place, there is a risk of undermining the confidence in the outputs.
Statements that do not reach the required threshold present an opportunity to engage in further discussion with experts. If consensus is close, those who originally leant away from it may adjust their perspectives based on the consolidated feedback from the first survey round. If consensus is falling significantly short, this is an opportunity to consider amending the statement according to first-round responses. Under either condition a new survey can then be issued. However, if consensus remains elusive after multiple iterations, it is prudent to accept non-consensus and report it accordingly. This holds evidential value too and should be acknowledged.
5. Achieving Publication and Impact for a Delphi Consensus
Attaining peer-reviewed publication following a Delphi Consensus is not always assured, although it is often necessary to establish credibility and help facilitate transformative change. Failing to diligently pursue this recognition runs the risk of the study going unnoticed and having no real impact.
To enhance the likelihood of successful publication, it is imperative to employ high-quality, scientifically precise writing. Recommendations should be actionable and underpinned by the evidence developed via the expert consensus and care should be taken not to overstate (e.g. a recommendation based on a statement that achieved 98% consensus is more confidently made than on a statement with 75%).
Additionally, transparency is paramount. While there are well documented strengths of Delphi Consensus as a research approach and when designed well can be considered scientifically robust, there will be limitations. Understanding these, recognising where in the process the risk of bias may be a concern and sharing detail around how this was mitigated is well appreciated by peer-review teams at well-regarded journals.
In tandem with publishing the study, effective communication materials that convey the Delphi Consensus results through various channels can substantially enhance a campaign’s efficacy. Activities such as poster presentations, video abstracts, Consensus summaries and slide kits are just a few examples of communication tools that help amplify impact.
Discover How Triducive Can Help
Triducive are leaders in healthcare for generating real-world evidence through Delphi Consensus. Our team, combining healthcare experts and scientific writers are able to help you at any stage of your evidence generation process. Contact us for more information about Delphi studies.