How to Measure the Impact of a Delphi Study
In short, the impact of a Delphi study is the change it enables. This may include influencing clinical practice, informing guidelines, shaping research priorities, supporting healthcare policy, improving clinical trial design or ultimately improving patient outcomes.
The Delphi method can turn expert knowledge into consensus on an important clinical or research question. But achieving consensus is only the beginning. When organisations ask “what is the impact of a Delphi study?”, the answer lies in what happens after consensus has been reached.
Understanding impact is important because organisations invest substantial time and resources in Delphi studies, and consensus alone does not guarantee meaningful change.
Consensus recommendations can inform clinical guidelines and practice, help shape healthcare policy, inform clinical development, identify research priorities and help define meaningful outcomes and endpoints. But turning expert agreement into real-world change requires effective dissemination, stakeholder engagement, adoption, implementation and evaluation.
What “impact” looks like will depend on the purpose of the study. It might mean recommendations being incorporated into guidelines, agreed endpoints being adopted in clinical trials, or consensus helping to address an unmet need during product development.
The key distinction is simple:
Consensus is the output. The impact of a Delphi study is what happens when that output is then used.
-
What is the impact of a Delphi study and how should it be defined?
Decide what impact should look like before the consensus study begins: what is this study intended to change or enable?
For example, is the aim to:
- support a change in clinical practice?
- inform guideline development?
- reduce variation in care?
- address an evidence gap?
- establish research priorities?
- determine important clinical trial endpoints?
- inform product development?
- support healthcare policy?
The answer will determine who needs to be engaged, how the findings should be disseminated and which measures will demonstrate real change.
This also helps ensure that the Delphi questions and final outputs are designed around a meaningful decision or problem, rather than simply around achieving agreement.
-
Measuring whether the findings reached the right people
Consensus cannot influence practice if the people who need to act on it never see it.
Publication in a peer-reviewed journal is often an important part of dissemination, providing a permanent and citable record of the findings.
Then, depending on the audience and objectives, dissemination might include:
- presentations at medical congresses
- conference posters and oral presentations
- webinars and educational meetings
- expert-led discussions
- scientific communications
- professional society engagement
- concise summaries of key recommendations
- digital and professional channels
For example, a Delphi study establishing recommendations for the management of a disease may be published in a relevant journal and presented at a major clinical congress. Expert participants can help communicate the findings to their professional networks, while targeted engagement can bring the recommendations to the attention of guideline developers and other clinical stakeholders.
The impact of a Delphi study at this stage can be assessed through measures such as reach, engagement and awareness.
However, a widely viewed presentation does not necessarily mean that practice has changed. Dissemination is the step that makes the subsequent impact of a Delphi study possible.
-
Measuring adoption, implementation and changes in clinical practice
Dissemination creates awareness, but real-world impact depends on whether the findings are adopted and put into practice.
For a clinical consensus study, adoption might mean recommendations are incorporated into clinical guidelines, endorsed by professional societies, included in institutional protocols or care pathways, or referenced in policy and educational resources. For a research-focused study, it might mean agreed outcomes or priorities are incorporated into clinical trials, research programmes or evidence-generation strategies.
This is where stakeholder engagement becomes important. Identifying guideline developers, clinical leaders, researchers and policymakers early allows the findings to be communicated in a way that supports their decision-making.
Implementation may then involve education and training, decision-support tools, local adaptation, multidisciplinary engagement, audit and feedback.
The resulting impact can be measured through:
- adherence to recommendations
- changes in diagnostic or treatment decisions
- changes in referral or monitoring practices
- use of recommended tools or outcomes
- consistency of care across settings
- changes in research practice
The measures should reflect the original objective. If the aim is to reduce variation in care, for example, the key measure may be whether clinical practice has become more consistent.
Consensus tells us what experts agree should happen. Adoption and implementation tell us whether that agreement is translated into practice.
Importantly, clinical impact is rarely attributable to the Delphi study alone. The consensus is one component of a wider pathway involving evidence, guidelines, implementation and clinical decision-making.
-
Measuring impact on research and clinical development
Consensus studies can also have an important impact before or alongside clinical practice by influencing research priorities, clinical development and evidence generation.
During the development of a medicine, device or other healthcare intervention, there may be uncertainty around unmet needs, patient populations, meaningful outcomes, research priorities or the evidence required to support clinical decision-making. A consensus study can help address these questions by bringing together relevant experts and stakeholders.
For example, a Delphi study may help determine:
- which patient characteristics are most important
- which outcomes are clinically meaningful
- which endpoints should be prioritised
- how outcomes should be defined
- which time points are appropriate for measurement
- which evidence gaps should be addressed
- which research questions should be prioritised
- which outcomes should be consistently measured, for example through the development of a core outcome set
The downstream impact may include changes to study or clinical trial design, the selection of endpoints, adoption of a core outcome set, greater consistency in outcome measurement, new research or evidence-generation programmes, funding decisions, subsequent publications or guideline development.
In this context, impact can be assessed by examining whether the consensus findings influenced what researchers chose to investigate, how subsequent studies were designed, and which evidence was generated.
-
Measuring the ultimate impact on patient outcomes
At the highest level, consensus may contribute to changes in patient outcomes and the overall quality and efficiency of care.
These effects are usually indirect. A consensus study may first influence clinical recommendations, research priorities, care pathways or professional practice, which may then contribute to changes in the care patients receive.
Potential impacts could include:
- improved quality of care
- improved patient experience
- earlier or more accurate diagnosis
- more appropriate or timely treatment
- reduced unwarranted variation in care
- improved clinical outcomes
- improved access to appropriate care
- more efficient use of healthcare resources
Measuring this level of impact can be challenging because patient outcomes are rarely attributable to a consensus study alone. Changes in clinical practice, healthcare policy, service organisation, new evidence, available treatments and many other factors may all contribute.
It may therefore be more appropriate to assess whether there is a credible pathway from the consensus findings to changes in practice and, ultimately, to patient outcomes. For example, consensus recommendations may be incorporated into a guideline, the guideline may influence clinical practice, and those changes in practice may subsequently lead to better or more consistent care.
The contribution of the consensus study should therefore be understood as part of a wider evidence-to-action pathway, rather than as a single intervention with a directly attributable effect on patient outcomes.
Frequently Asked Questions
What is the impact of a Delphi study?
The impact of a Delphi study is the change that occurs when consensus findings are adopted and used in practice, research, policy or clinical development.
How do you measure the impact of a Delphi study?
Impact can be measured through dissemination, implementation, adoption, influence on research and potential effects on patient outcomes.
Can a Delphi study inform clinical guidelines?
Yes. Consensus recommendations are frequently used to inform guideline development where evidence gaps exist.
Conclusion
Not every Delphi study needs to measure impact at every stage. The most appropriate measures will depend on the original purpose of the consensus study and the change it was intended to support.
The strongest consensus studies are therefore designed with their eventual application in mind. Thinking about impact from the outset helps ensure that the consensus process is not treated as a standalone research exercise, but as part of a broader pathway from evidence and expert knowledge to action.
Reaching consensus is an important milestone, but it is not the end point. The value of a Delphi study lies in what that agreement enables: it may inform clinical guidance, influence policy, improve consistency of care, shape clinical development, identify meaningful endpoints or establish priorities for future research.
Achieving these outcomes, however, requires a deliberate pathway from consensus to communication, engagement, adoption, implementation and, ultimately, evaluation.
The impact of a Delphi study should therefore not be judged simply by whether experts reached agreement, but by what happens because they did.
About Triducive
With over 70 manuscripts published in peer-reviewed journals, Triducive has extensive experience in developing and delivering communications strategies to amplify the results of a Delphi consensus.
Triducive supports organisations throughout the consensus journey, from study design and stakeholder engagement through to dissemination, implementation and evaluation of impact.
Contact us for more information about the Delphi method and amplification campaigns.