What Works in Promoting What Works?

Last week the PPIW presented at the What Works Global Summit. Representatives from over forty countries gathered at this major international conference to discuss the state of evidence-informed policy and practice globally, as well as future challenges and opportunities.  Our presentation on putting evidence to work for politicians in Wales drew in over thirty delegates from around the world who were very interested in the role of the PPIW and the lessons that we have learnt. We emphasised the importance of knowledge brokers in the relationship between evidence suppliers and users, and the need for those producing and translating evidence to understand both the policy process and prevailing political narratives. The interest our paper attracted highlighted that we should be talking about what we do and why more often, so watch this space!

The summit as a whole was both depressing and encouraging. Howard White, the Chief Executive Officer of the Campbell Collaboration, argued that research “is the most important thing we can do to help people, but research is good for nothing if it doesn’t make it into policy and practice”. While vast amounts of money are spent on research, there is still a startling lack of evidence use in policy and practice.  One presenter reported that 40% of World Bank reports have never been read and 87% have never been cited.

However, it was uplifting to hear from so many likeminded ‘evidence advocates’ and to learn that there are initiatives all over the world working to increase the use of evidence. It was also reassuring to hear the same key messages coming through many of the presentations. Given this widespread, common understanding of what matters in evidence informed policy and practice there is hope that the ‘evidence revolution’ can inspire a new culture of evidence use globally. Here we offer our reflections on some of the key messages from the summit and how the PPIW is contributing to and helping to shape learning about how to make evidence useful to policy makers in Wales.

 

  1. The importance of engagement, relationships and an understanding of systems

As we said in our presentation, it is increasingly recognised that passive approaches to the dissemination of research findings simply don’t work. The increase of intermediary organisations (or ‘knowledge brokers’) like the PPIW who mediate between the academic and policy-maker or practitioner audiences reflects the growing recognition that the traditional ‘generate – transmit – adopt’ model of knowledge utilisation, which suggests good quality research naturally flows into policy and practice, just isn’t true to life. Indeed, many members of the UK What Works Network, which was initially created to synthesise research on what works and judge evidence quality, are increasingly turning their attention to knowledge mobilisation activities as they realise that packaging and translating research in an accessible way is not enough to get it used.

The relationship between research and policy or practice is complicated and nuanced. To promote evidence use we need to pay attention not just to the quality of the evidence but also to how to engage users in designing, undertaking and reporting research. This means influencing professional norms and standards and creating networks of evidence producers and potential users. The PPIW is, of course, trying to play its part in Wales by facilitating engagement between Welsh Government Ministers and independent sources of expertise.

  1. The importance of strategic evidence planning

While it was great to meet so many people working on the same challenges, it was disconcerting to find that their efforts are often uncoordinated. One presenter pointed out that we have a proliferation of organisations working on and publishing synthesis reviews on the same topics using the same evidence but different evidence standards, resulting in multiple reviews on the same topics drawing different conclusions. Well-intentioned but uncoordinated efforts can result in users getting mixed messages and it was suggested that we therefore need international calls for evidence and a more strategic approach to knowledge gathering. Countries pooling their efforts to avoid repeating the same systematic reviews was seen as a way of overcoming this.

The potential for evidence gap maps to contribute to a more joined up approach to evidence informed policy and practice was also explored. Initially developed by 3ie, evidence gap maps provide thematic platforms for accessing existing evidence and a way of identifying evidence gaps to inform research commissioning. While there are challenges associated with these platforms, evidence gap maps could facilitate greater awareness and communication across evidence ecosystems in different subject areas.

Through our recent evidence needs workshop series, the PPIW has been helping to understand and identify the evidence that Welsh Ministers may need to tackle some of the big challenges facing Wales over the coming five years and we are keen to contribute to extend this by working with other countries.

 

  1. The importance of education and expectation management

Findings from research are rarely straightforward. Yet, decision makers want clear, actionable solutions to problems.  We need interventions which build decision makers’ capacity to access and make sense of evidence. But how do we get people to be comfortable with mixed results? How do we reconcile the need to deal in real world complexities (the nuances that come with research) and the need to communicate it to the general public (including only having 140 characters to summarise research on Twitter)?

The summit highlighted that we need to educate decision makers about the complexities and limitations that surround research in order to manage expectations. If one trial suggests a programme is effective, that doesn’t necessarily prove that it will work if rolled out at scale. This point further emphasises the importance of knowledge brokers who are able to translate and interpret research findings for decisions makers, like the role of the PPIW in Wales.

 

  1. The importance of context

Research use is context specific. As one delegate said, “it is important to understand the science of using science, but it is also important to recognise the art of using science; the nuances and context specific factors which make a difference to success”.  In social science, interventions from one setting can rarely be adopted unchanged in another. They need to be adapted for a new setting. This needs to be taken into account when using research, particularly when it comes to implementing findings. The PPIW is well aware of this challenge and we always seek to evaluate evidence in terms of its relevance to the Welsh context for policy-makers in Wales.

 

  1. The need for humility

Finally, though we and others try to make available the current best evidence, it is important to be humble and remember that there are limitations to what empirical evidence can tell us and that there are many other factors which (rightly) influence policy and practice. It is important that the empirical analysis which the PPIW and others provide is set alongside the ‘craft knowledge’ which teachers, clinicians, police officers and others have acquired through their work ‘on the frontline’.

 

We are at a pivotal point in discussions around evidence informed policy and practice. Collectively, we know many of the factors that shape evidence use and have experience of what works and what doesn’t in encouraging the uptake of evidence. What we need to focus on now is how to engage effectively with users of evidence. This calls for effective processes to translate evidence and mutual understanding between producers and users. We need to build relationships between policy experts and policy makers in a more structured and coordinated way and it is encouraging to know that others are recognising this and trying to do the same as we, at the PPIW, are seeking to achieve in Wales.

Comments are closed