Better Data for Better Decisions: The Evidence Ecosystem

The evidence movement is achieving real progress. Five new ‘What Works Centres’, in crime reduction, local economic growth, education, ageing better and early intervention have now opened and are linked in a What Works Network, of which the PPIW is a part. The centres’ overall aim is to improve the use of high quality evidence when government makes decisions about public services.

These new arrangements in very different sectors should be thought of as components of evidence ecosystems. After all, the creation and implementation of effective policies and interventions, in whichever policy or service area they serve, depend on a process of evidence generation, synthesis and evidence-informed behaviour change.

This is the basis for my report to the Cabinet Office, ‘How to achieve more effective services: the evidence ecosystem’. This report is based on my research to identify and link the elements of the evidence supply chain into a coherent ecosystem; to identify gaps and barriers to evidence flow and implementation; and to provide practical recommendations on the basis of which the evidence ecosystem can be improved.

As in all efficient systems, form should follow function. Not only must each stage of the process be operational but stages must also be connected. No amount of evidence synthesis will achieve much if relevant evidence has not been generated. Neither will the best evidence informed guidance achieve its purpose if the means to promote and adopt it are missing.

The oft heard reference to evidence ‘pipelines’ reinforces this need for a systems approach and prompts further consideration of this oil industry analogy. Evidence needs to flow through the ecosystem from generation to end user – it will not flow on its own. As in the petroleum industry, ‘pumps’, or in marketing language, ‘product push’ and ‘demand pull’ are needed at every stage. A system cannot function without conduits connecting each stage.

Evidence is often categorised according to evaluation method (randomised trials, qualitative approaches etc.), but the petroleum industry offers an additional quality: usability. Practitioners and commissioners need evidence in usable forms. Evidence refinement is therefore essential.

This systems perspective together with interviews of key personnel across what works sectors led to a series of recommendations designed to address both structural and functional problems. In some sectors there are major capacity shortfalls, especially for evidence generation. In others there are disconnects, for example, between academics and front line services. In others still, the workforce needs to be categorised so that target groups for training can be identified. It is clear that raw and synthesised evidence in the forms in which it is currently published is often unusable – and sometimes does more harm than good because it obscures and generates scepticism about useful evidence. Recommendations in my report include:

Evidence creation

A healthy ecosystem generates evidence using appropriate methods. Whilst randomised and quasi-experimental methods will not always be suitable, when they are used well they are the most definitive and least equivocal way of demonstrating impact and should be an integral and indispensable part of evidence-based policy making. This should be reflected in evidence quality standards developed by the What Works Network. To increase trials and trials skills capacity, the ESRC should develop a trials unit and should identify ways in which What Works sectors can be represented on its council.

Evidence translation

Institutions and roles that provide more than one function help connect the entire system; opportunities to develop these should be explored. Examples include the Education

Endowment Foundation (EEF), which both funds evidence production and synthesises evidence, and clinician-scientists in medicine who both generate and implement evidence. When combined with public service vocation, the curiosity of researchers equipped with experimental skills is the beating heart of functional evidence ecosystems. By the same token, there are likely to be few acts more damaging to systems than separating researchers and practitioners.

Evidence-informed guidelines and new evidence should be published in short, accessible formats; extensive use of social media and service magazines as sources of evidence across all What Works sectors reflects the urgent need for this targeted approach to evidence promotion.

The What Works brand should be strengthened to increase evidence visibility, identity and authority. The successful National Institute for Health and Care Excellence (NICE) accreditation model could be adapted across What Works sectors to signify quality assurance.

Evidence implementation

What Works Centres should, with national organisations responsible for service quality and with commissioners and regulators, explore and implement ways in which they can increase traction in their sectors.

Research assistants search for and summarise evidence for commissioners across sectors; they should be targeted for evidence skills training. Such training should also be made widely available to policy professionals in Welsh Government and Whitehall and be integrated into the commissioning skills training and development supported by the Cabinet Office.

Local, face-to-face meetings are hugely important for creating an environment in which professionals feel compelled and supported to engage with and act on the basis of evidence. These learning groups should be developed and maintained in all sectors; professional bodies have an important role to play in this context.

Conclusion

It is becoming clear that evidence ecosystems are an essential foundation of good government. Service improvement is much more difficult and less efficient without them. But they comprise many components and depend on many different agencies.  Constant vigilance is necessary both to identify and fill system gaps and to identify and rectify faults.


 

About the author: Jonathan Shepherd is Professor of oral and maxillofacial surgery and Director of the violence and society research group at Cardiff University. He is the independent member of the Cabinet Office What Works Council and represents the Academy of Medical Sciences on the Home Office Science Advisory Council. 

Comments are closed