case study research for better evaluations of complex interventions rationale and CORD-Papers-2022-06-02 (Version 1)

Title: Case study research for better evaluations of complex interventions: rationale and challenges
Abstract: BACKGROUND: The need for better methods for evaluation in health research has been widely recognised. The complexity turn has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether and under which conditions interventions in complex systems improve health services or the public health and what mechanisms might link interventions and outcomes. We argue that case study researchcurrently denigrated as poor evidenceis an under-utilised resource for not only providing evidence about context and transferability but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear. MAIN BODY: Case study research as an overall approach is based on in-depth explorations of complex phenomena in their natural or real-life settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second there is little consensus about what a case study is and considerable diversity in how empirical case studies are conducted and reported. Finally as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail) it can be challenging to identify the key messages related to intervention evaluation from case study reports. CONCLUSION: Whilst the diversity of published case studies in health services and public health research is rich and productive we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.
Published: 2020-11-10
Journal: BMC Med
DOI: 10.1186/s12916-020-01777-6
DOI_URL: http://doi.org/10.1186/s12916-020-01777-6
Author Name: Paparini Sara
Author link: https://covid19-data.nist.gov/pid/rest/local/author/paparini_sara
Author Name: Green Judith
Author link: https://covid19-data.nist.gov/pid/rest/local/author/green_judith
Author Name: Papoutsi Chrysanthi
Author link: https://covid19-data.nist.gov/pid/rest/local/author/papoutsi_chrysanthi
Author Name: Murdoch Jamie
Author link: https://covid19-data.nist.gov/pid/rest/local/author/murdoch_jamie
Author Name: Petticrew Mark
Author link: https://covid19-data.nist.gov/pid/rest/local/author/petticrew_mark
Author Name: Greenhalgh Trish
Author link: https://covid19-data.nist.gov/pid/rest/local/author/greenhalgh_trish
Author Name: Hanckel Benjamin
Author link: https://covid19-data.nist.gov/pid/rest/local/author/hanckel_benjamin
Author Name: Shaw Sara
Author link: https://covid19-data.nist.gov/pid/rest/local/author/shaw_sara
sha: 74ac1e8601c01f9e958c222903095476598ed99b
license: cc-by
license_url: https://creativecommons.org/licenses/by/4.0/
source_x: Medline; PMC
source_x_url: https://www.medline.com/https://www.ncbi.nlm.nih.gov/pubmed/
pubmed_id: 33167974
pubmed_id_url: https://www.ncbi.nlm.nih.gov/pubmed/33167974
pmcid: PMC7652677
pmcid_url: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7652677
url: https://www.ncbi.nlm.nih.gov/pubmed/33167974/ https://doi.org/10.1186/s12916-020-01777-6
has_full_text: TRUE
Keywords Extracted from Text Content: research-currently evidence-is locales p126 Canguilhem UK patient JG COVID-19 Hyett London, Guba's [47] and Stake's [33 UK's Medical Research Council within-case Savransky [16 subject/purpose/approach/process patients MP SS TG CP
Extracted Text Content in Record: First 5000 Characters:Background: The need for better methods for evaluation in health research has been widely recognised. The 'complexity turn' has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research-currently denigrated as poor evidence-is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear. Main body: Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports. Conclusion: Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences. The need for methodological development to address the most urgent challenges in health research has been welldocumented. Many of the most pressing questions for public health research, where the focus is on system-level determinants [1, 2] , and for health services research, where provisions typically vary across sites and are provided through interlocking networks of services [3] , require methodological approaches that can attend to complexity. The need for methodological advance has arisen, in part, as a result of the diminishing returns from randomised controlled trials (RCTs) where they have been used to answer questions about the effects of interventions in complex systems [4] [5] [6] . In conditions of complexity, there is limited value in maintaining the current orientation to experimental trial designs in the health sciences as providing 'gold standard' evidence of effect. There are increasing calls for methodological pluralism [7, 8] , with the recognition that complex intervention and context are not easily or usefully separated (as is often the situation when using trial design), and that system interruptions may have effects that are not reducible to linear causal pathways between intervention and outcome. These calls are reflected in a shifting and contested discourse of trial design, seen with the emergence of realist [9] , adaptive and hybrid (types 1, 2 and 3) [10, 11] trials that blend studies of effectiveness with a close consideration of the contexts of implementation. Similarly, process evaluation has now become a core component of complex healthcare intervention trials, reflected in MRC guidance on how to explore implementation, causal mechanisms and context [12] . Evidence about the context of an intervention is crucial for questions of external validity. As Woolcock [4] notes, even if RCT designs are accepted as robust for maximising internal validity, questions of transferability (how well the intervention works in different contexts) and generalisability (how well the intervention can be scaled up) remain unanswered [5, 13] . For research evidence to have impact on policy and systems organisation, and thus to improve population and patient health, there is an urgent need for better methods for strengthening external validity, including a better understanding of the relationship between intervention and context [14] . Policymakers, healthcare commissioners and other research users require credible evidence of relevance to their settings and populations [15] , to perform what Rosengarten and Savransky [16] call 'careful abstraction' to the locales that matter for them. They also require robust evidence for
Keywords Extracted from PMC Text: patient UK 's COVID-19 UK's Stake's [33]—are subject/purpose/approach/process Canguilhem p126 Guba's London, locales within-case patients Savransky [16 Hyett [38]—rather JG
Extracted PMC Text Content in Record: First 5000 Characters:The need for methodological development to address the most urgent challenges in health research has been well-documented. Many of the most pressing questions for public health research, where the focus is on system-level determinants [1, 2], and for health services research, where provisions typically vary across sites and are provided through interlocking networks of services [3], require methodological approaches that can attend to complexity. The need for methodological advance has arisen, in part, as a result of the diminishing returns from randomised controlled trials (RCTs) where they have been used to answer questions about the effects of interventions in complex systems [4–6]. In conditions of complexity, there is limited value in maintaining the current orientation to experimental trial designs in the health sciences as providing 'gold standard' evidence of effect. There are increasing calls for methodological pluralism [7, 8], with the recognition that complex intervention and context are not easily or usefully separated (as is often the situation when using trial design), and that system interruptions may have effects that are not reducible to linear causal pathways between intervention and outcome. These calls are reflected in a shifting and contested discourse of trial design, seen with the emergence of realist [9], adaptive and hybrid (types 1, 2 and 3) [10, 11] trials that blend studies of effectiveness with a close consideration of the contexts of implementation. Similarly, process evaluation has now become a core component of complex healthcare intervention trials, reflected in MRC guidance on how to explore implementation, causal mechanisms and context [12]. Evidence about the context of an intervention is crucial for questions of external validity. As Woolcock [4] notes, even if RCT designs are accepted as robust for maximising internal validity, questions of transferability (how well the intervention works in different contexts) and generalisability (how well the intervention can be scaled up) remain unanswered [5, 13]. For research evidence to have impact on policy and systems organisation, and thus to improve population and patient health, there is an urgent need for better methods for strengthening external validity, including a better understanding of the relationship between intervention and context [14]. Policymakers, healthcare commissioners and other research users require credible evidence of relevance to their settings and populations [15], to perform what Rosengarten and Savransky [16] call 'careful abstraction' to the locales that matter for them. They also require robust evidence for understanding complex causal pathways. Case study research, currently under-utilised in public health and health services evaluation, can offer considerable potential for strengthening faith in both external and internal validity. For example, in an empirical case study of how the policy of free bus travel had specific health effects in London, UK, a quasi-experimental evaluation (led by JG) identified how important aspects of context (a good public transport system) and intervention (that it was universal) were necessary conditions for the observed effects, thus providing useful, actionable evidence for decision-makers in other contexts [17]. The overall approach of case study research is based on the in-depth exploration of complex phenomena in their natural, or 'real-life', settings. Empirical case studies typically enable dynamic understanding of complex challenges rather than restricting the focus on narrow problem delineations and simple fixes. Case study research is a diverse and somewhat contested field, with multiple definitions and perspectives grounded in different ways of viewing the world, and involving different combinations of methods. In this paper, we raise awareness of such plurality and highlight the contribution that case study research can make to the evaluation of complex system-level interventions. We review some of the challenges in exploiting the current evidence base from empirical case studies and conclude by recommending that further guidance and minimum reporting criteria for evaluation using case studies, appropriate for audiences in the health sciences, can enhance the take-up of evidence from case study research. Well-conducted and described empirical case studies provide evidence on context, complexity and mechanisms for understanding how, where and why interventions have their observed effects. Recognition of the importance of context for understanding the relationships between interventions and outcomes is hardly new. In 1943, Canguilhem berated an over-reliance on experimental designs for determining universal physiological laws: 'As if one could determine a phenomenon's essence apart from its conditions! As if conditions were a mask or frame which changed neither the face nor the picture!' ([18] p126). More recently, a concern with context has been expressed
PDF JSON Files: document_parses/pdf_json/74ac1e8601c01f9e958c222903095476598ed99b.json
PMC JSON Files: document_parses/pmc_json/PMC7652677.xml.json
G_ID: case_study_research_for_better_evaluations_of_complex_interventions_rationale_and