In my first week as an undergraduate we were given an assignment on the relationship between super-national bodies, such as the EU, and devolution in the United Kingdom. At the time, I didn’t get it. These seemed to me to be divergent political trends pulling in opposite directions. And I favoured internationalism over parochialism.
I hope the years have made me a bit wiser. At the political level I see how a super-national body can strengthen sub-national political identity as a counter to the dominant country in unequal national unions. And I am firmly behind adapting policy and practice to local context. But most of all I see how super-national structures can support local ones in bringing a broader range of knowledge and experience to bear in formulating local policy.
The re-emergence of a distinct Welsh polity since the late 1800s, consolidated by devolution – most notably the establishment of the National Assembly in 1998 – means that policies in Wales need not be the same as those in England. Welsh policy and practice should be policy and practice appropriate for the Welsh context and Welsh culture. However, as Emily St.Denny wrote in her recent PPIW blog, the ‘Made in Wales’ approach to date has been more about different ways of doing policy rather than different policies. There are constraints on being too different, such as party policy of the dominant UK parties and a concern of being criticized for having different policies to England in case outcomes are worse than those in England – though the divergence in outcomes may be nothing to do with the policy.
But the growth of evidence-based policy and practice in recent years provides opportunities to break away toward greater diversity in policy and practice. There is growing awareness – and PPIW is playing a part in raising this awareness in Wales – that there is scant evidence behind many policies and programmes. And so there is a search for evidence-based approaches. In carrying out this search, Welsh policy makers should look further than just over the border.
Global knowledge should be brought to bear in deciding what to do. This doesn’t mean ‘it worked in Chicago so let’s do it in Cardiff’. It doesn’t mean that for two important reasons. First, we should not rely on single studies. We should use systematic reviews which summarize all high quality evidence about the effectiveness of a particularly policy, programme or practice. Second, we use global evidence to decide what to try and test locally. The Campbell Collaboration is an international research network which publish reviews which summarize global knowledge in this way.
Campbell reviews show that some things haven’t worked anywhere so it is probably best not to waste time on them. Harsh regimes for juveniles to deter them from crime, such as Scared Straight and Boot Camps, have not been successful. Indeed, Scared Straight is more likely to lead young people to a life of crime than the opposite. Curfews for young people are similarly ineffective in reducing crime.
But more usually reviews will identify which programmes seem promising. The Campbell review of anti-bullying programmes in schools identifies seven approaches which seem effective. So local adaptation mean seeing which of those seven seem to fit the local context. Best to pick two or three and then test them locally. The key here is local testing. Evidence-based policy is not a blueprint approach. It is not a ‘it worked there let’s do it here approach’. It is ‘it worked there, let’s try it and test it here’. If not ‘Made in Wales’, then ‘Tested in Wales’.
Sharing experience of what works is of course a two-way street. There are already examples of UK-wide policies which started in Wales, such as charging for plastic bags. As the Welsh evidence agenda grows, with more on more policy and practice put to the test, these studies will enter the global database of knowledge of what works in Campbell reviews and so contribute to better lives around the world.
Knowledge intermediaries such as PPIW have an important part to play in this process. Part of this role is awareness raising amongst politician, practitioners, the media and the public about the importance of evidence and different types of evidence. Most politicians don’t have the training to distinguish good evidence from bad, and unfortunately incentives in the media are toward sensation rather than science. Knowledge brokers should teach critical evidence appraisal.
And, of course, knowledge brokers help policy makers sift global knowledge. Systematic reviews are written for researchers not policy makers. Knowledge brokers help policy makers interpret and apply this knowledge by producing locally-relevant intermediate products, like policy briefs, seminars and workshops and other forms of engagement.
So PPIW is a key agency in the local application of global knowledge for a better future for Wales. And Campbell is key agency at the global level maintaining the evidence base which makes the work of groups such as PPIW possible.
About the author: Howard White is the CEO of the Campbell Collaboration. Previously he was the founding Executive Director of the International Initiative for Impact Evaluation (3ie) and before that led the impact evaluation programme of the World Bank’s Independent Evaluation Group. He started his career as an academic researcher at the Institute of Social Studies in the Hague, and the Institute of Development Studies, University of Sussex. As an academic he leans toward work with policy relevance, and working in the policy field leans toward academic rigour as a basis for policy and practice.