qualitative process evaluation from a complex systems perspective a systematic review CORD-Papers-2022-06-02 (Version 1)

Title: Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators.
Abstract: BACKGROUND Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A complex systems approach has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations. METHODS AND FINDINGS We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1 2014-September 30 2019 (Scopus MEDLINE Web of Science) citation searching and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach integrated qualitative methods reported empirical findings and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in for example urban planning sexual health violence prevention substance use and community transformation. Evaluations were conducted in settings such as schools workplaces and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations the consideration of complex systems influenced intervention design evaluation planning or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from or co-occurring with intervention implementation over time. Limitations to our approach are including only English-language papers reliance on study authors reporting their utilization of complex systems concepts and subjective judgment from the reviewers relating to which concepts featured in each study. CONCLUSION This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach.
Published: 2020-11-01
Journal: PLoS medicine
DOI: 10.1371/journal.pmed.1003368
DOI_URL: http://doi.org/10.1371/journal.pmed.1003368
Author Name: McGill Elizabeth
Author link: https://covid19-data.nist.gov/pid/rest/local/author/mcgill_elizabeth
Author Name: Marks Dalya
Author link: https://covid19-data.nist.gov/pid/rest/local/author/marks_dalya
Author Name: Er Vanessa
Author link: https://covid19-data.nist.gov/pid/rest/local/author/er_vanessa
Author Name: Penney Tarra
Author link: https://covid19-data.nist.gov/pid/rest/local/author/penney_tarra
Author Name: Petticrew Mark
Author link: https://covid19-data.nist.gov/pid/rest/local/author/petticrew_mark
Author Name: Egan Matt
Author link: https://covid19-data.nist.gov/pid/rest/local/author/egan_matt
license: cc-by
license_url: https://creativecommons.org/licenses/by/4.0/
source_x: Medline
source_x_url: https://www.medline.com/
pubmed_id: 33137099
pubmed_id_url: https://www.ncbi.nlm.nih.gov/pubmed/33137099
url: https://doi.org/10.1371/journal.pmed.1003368 https://www.ncbi.nlm.nih.gov/pubmed/33137099/
has_full_text: FALSE
G_ID: qualitative_process_evaluation_from_a_complex_systems_perspective_a_systematic_review