Hilary Thomson has been highlighting the need for improved understanding, management and transparency of narrative synthesis methods during a recent six week sojourn through South Korea, Japan and Australia. Workshops at the Seoul Colloquium and meetings in Melbourne proved ideal forums to discuss and debate critical issues and methodological challenges related to narrative synthesis, which to date have received little collective attention.
Narrative synthesis has become a key area of focus for Hilary in both her role as a co-editor with Cochrane Public Health and as a Senior Investigator Scientist at the University of Glasgow’s Social and Public Health Sciences Unit. Her interest is informed by over two decades of experience developing primary research and complex systematic reviews assessing the health and socio-economic impacts of housing improvement and area regeneration, welfare reform, transport and employment initiatives in the UK and beyond.
‘I’m really interested in narrative synthesis as a means to improve the rigour and reliability of complex reviews and our evidence base more broadly,’ Hilary explains. ‘While people have different ideas about what narrative synthesis is, put simply it’s the method used when you can’t analyse data using meta‐analysis or can only pool some of the included studies and/or data statistically, and so the synthesis of all the included studies needs to be presented textually. It’s also used for data from different study designs that aren’t suitable for lumping all together in analysis or where you may have captured a very wide range of interventions. So it’s used quite commonly in systematic reviews, including around 20 per cent of Cochrane Reviews, but at the same time it’s not universally well understood or applied as a method.’
‘In practical terms, one of the main problems we find is that results can be very opaque because most authors don’t report how they have conducted the narrative synthesis. Rather, the conclusions are presented but you can’t really see where they’ve come from and how they are supported by the data - and that’s totally against the principles of systematic review. Sometimes it can be done well, but given the methods are not reported it’s very difficult to assess how reliable that synthesis is. And this is one of the main issues we’re working to address.’
As part of this undertaking, Hilary is currently working with Joanne McKenzie and Sue Brennan from Cochrane Australia and Rebecca Ryan from the Cochrane Consumers and Communication Group on new Cochrane Handbook chapters focused on non-statistical methods of synthesis. The chapters are designed to provide greater clarity and guidance for authors who are grappling with narrative synthesis in complex reviews. Narrative synthesis is a method most commonly used to synthesise complex quantitative data in systematic reviews published by the Public Health, Consumers and Communication, and Effective Practice and Organisation of Care Groups.
‘Developments to improve review methods have largely overlooked narrative synthesis to date, and consensus is lacking about how quantitative data which cannot be meta-analysed should be synthesised in a transparent way. As a result, it can be quite difficult to assess rigour and potential bias in narrative synthesis,’ Hilary says. ‘So there’s a critical need for a program of methodological development to underpin and improve narrative synthesis of quantitative data. This is the kind of thing that has been done for other methods. For example 20 years ago RCTs were not reported well. Even just by flagging that narrative synthesis is an issue that people are not really dealing with well, and opening up the debate about what we can do about it, we’re hoping to see greater recognition and inspire a more rigorous approach. I think many people don’t really consider narrative synthesis as a method and perhaps don’t see the potential it holds. We’ll be working toward changing that.’
Image: Hilary Thomson, Sue Brennan and Joanne McKenzie talk narrative synthesis in Seoul