strengthening routine health data analysis in ethiopia the operational research and CORD-Papers-2022-06-02 (Version 1)

Title: Strengthening routine health data analysis in Ethiopia: the Operational Research and Coaching for Analysts (ORCA) experience
Abstract: Many routine health information systems (RHIS) show persistent gaps between recording and reporting data and their effective use in solving problems. Strengthening RHIS has become a global priority to track and address national health goals. In Ethiopia the Ministry of Health and Bill & Melinda Gates Foundation introduced the Operational Research and Coaching for Analysts (ORCA) capacity development project co-designed with the London School of Hygiene & Tropical Medicine which delivered training coaching and mentoring support. We present the development experiences and perceptions of ORCA as a mechanism to enhance data quality analysis interpretation and use. ORCA integrated capacity development activities into national data analysts routine workload over a period of 2 years. Participating analysts were drawn from across the Ministry of Health directorates and two of its closely aligned agencies: the Ethiopian Public Health Institute and the Ethiopian Pharmaceutical Supply Agency. We used mixed methods (knowledge questionnaire semi-structured interviews programme records) to document the fidelity feasibility reach and acceptability of ORCA and identify early signs of improved knowledge and changing institutional practices. Thirty-six participants completed the programme. Working in interdisciplinary groups on specific national health indicators they received training workshops and support for study design fieldwork and analysis to build skills in assessing data quality and interpreting findings relevant to policy. Personal development grants and laptops provided incentives for sustained engagement. Participants appreciated ORCAs applied and practical approach as well as good communication from administrators and clear links to national strategy. They also expressed frustration with delays difficulties prioritising project work over routine responsibilities and lack of formal accreditation. Knowledge and analytic skills increased and participants were able to integrate experiences from the project into their future work. Health system managers saw potential in longer-term improvements in data analysis and application to policy although no clear changes were observed yet.
Published: 2021-03-31
Journal: Global health action
DOI: 10.1080/16549716.2021.1901390
DOI_URL: http://doi.org/10.1080/16549716.2021.1901390
Author Name: Busza Joanna
Author link: https://covid19-data.nist.gov/pid/rest/local/author/busza_joanna
Author Name: Lemma Seblewengel
Author link: https://covid19-data.nist.gov/pid/rest/local/author/lemma_seblewengel
Author Name: Janson Annika
Author link: https://covid19-data.nist.gov/pid/rest/local/author/janson_annika
Author Name: Adem Serawit Omar
Author link: https://covid19-data.nist.gov/pid/rest/local/author/adem_serawit_omar
Author Name: Berhanu Della
Author link: https://covid19-data.nist.gov/pid/rest/local/author/berhanu_della
Author Name: Defar Atkure
Author link: https://covid19-data.nist.gov/pid/rest/local/author/defar_atkure
Author Name: Persson Lars ke
Author link: https://covid19-data.nist.gov/pid/rest/local/author/persson_lars_ke
Author Name: Kllestl Carina
Author link: https://covid19-data.nist.gov/pid/rest/local/author/kllestl_carina
sha: b6534ebde4f61bed3b5ae637957e484c109ce0eb
license: cc-by
license_url: https://creativecommons.org/licenses/by/4.0/
source_x: Medline; PMC
source_x_url: https://www.medline.com/https://www.ncbi.nlm.nih.gov/pubmed/
pubmed_id: 33789545
pubmed_id_url: https://www.ncbi.nlm.nih.gov/pubmed/33789545
pmcid: PMC8018453
pmcid_url: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8018453
url: https://doi.org/10.1080/16549716.2021.1901390 https://www.ncbi.nlm.nih.gov/pubmed/33789545/
has_full_text: TRUE
Keywords Extracted from Text Content: Bill participants participants [12] [13] [14] Laptops donors EPSA people BMGF SARS-CoV-2 midline EBP lowto-mid joint Bill DHIS-2 end-line EPHI Bill PI EPHI CK
Extracted Text Content in Record: First 5000 Characters:Many routine health information systems (RHIS) show persistent gaps between recording and reporting data and their effective use in solving problems. Strengthening RHIS has become a global priority to track and address national health goals. In Ethiopia, the Ministry of Health and Bill & Melinda Gates Foundation introduced the Operational Research and Coaching for Analysts (ORCA) capacity development project, co-designed with the London School of Hygiene & Tropical Medicine, which delivered training, coaching and mentoring support. We present the development, experiences, and perceptions of ORCA as a mechanism to enhance data quality, analysis, interpretation and use. ORCA integrated capacity development activities into national data analysts' routine workload over a period of 2 years. Participating analysts were drawn from across the Ministry of Health directorates and two of its closely aligned agencies: the Ethiopian Public Health Institute and the Ethiopian Pharmaceutical Supply Agency. We used mixed methods (knowledge questionnaire, semi-structured interviews, programme records) to document the fidelity, feasibility, reach, and acceptability of ORCA and identify early signs of improved knowledge and changing institutional practices. Thirty-six participants completed the programme. Working in interdisciplinary groups on specific national health indicators, they received training workshops and support for study design, fieldwork, and analysis to build skills in assessing data quality and interpreting findings relevant to policy. Personal development grants and laptops provided incentives for sustained engagement. Participants appreciated ORCA's applied and practical approach as well as good communication from administrators and clear links to national strategy. They also expressed frustration with delays, difficulties prioritising project work over routine responsibilities, and lack of formal accreditation. Knowledge and analytic skills increased and participants were able to integrate experiences from the project into their future work. Health system managers saw potential in longer-term improvements in data analysis and application to policy, although no clear changes were observed yet. ARTICLE HISTORY The United Nations Development Programme defines capacity development as 'the process through which individuals, organizations and societies obtain, strengthen and maintain the capabilities to set and achieve their own development objectives over time' [1] . Routine health information systems (RHIS) are one part of the health system that has become a global priority for capacity development initiatives as a means to improve and monitor national health outcomes and goals [2, 3] . RHIS in many low-income countries remain weak and underutilised for evidence-based decision-making [4] . In response, governments and donors recommend efforts to improve RHIS quality focused on data collection, use, and processes [5] . Health system improvements are only sustained once embedded into institutional culture, processes and practices so that they are resilient to changes in leadership, staffing and policies over time [6] . Strengthening capacity is multi-dimensional, comprising a mix of technical assistance, training and skill-building, new operational tools, peer networking, and incentives. It is multi-level, targeting national health architecture, organisational management, and frontline staff performance. RHIS capacity strengthening frequently centres around new technologies (e.g. the widely adopted DHIS-2) and technical skills (data entry, transfer, aggregation, synthesis, analysis, and interpretation). Capacity strengthening often consists solely of training, lacking deeper engagement with whole health systems [7] . Few capacity development efforts are evaluated [8] , although evidence is accumulating to suggest that they are most likely to succeed if they involve genuine country ownership and investment in the process, utilise problem-based 'on the job' learning, and embed technical assistance within existing health system structures and practices [6, 9, 10] . Ultimately, the need for RHIS capacity strengthening is highlighted by persistent gaps between recording and reporting information and its applied use in solving problems [4] . Functioning RHIS systems should support action rather than data analysis for its own sake, so that each type of data collected has a clear purpose and is appropriately interpreted to inform policies and programmes that improve health outcomes at population level [11] . Ethiopia has put RHIS quality at the forefront of its health agenda. Recognising serious problems of routine data quality [12] [13] [14] , the Government of Ethiopia included strengthening RHIS, in Ethiopia called the routine Health Management Information System (HMIS) within its 'Information Revolution' in the first Health Sector Transformation Plan of Ethiopia [15] . The Ethiopian Ministry of Health (MOH), prioritis
Keywords Extracted from PMC Text: EPSA end-line EBP joint EPHI participants SARS-CoV-2 ... low-to-mid people ... people midline BMGF 's
Extracted PMC Text Content in Record: First 5000 Characters:The ORCA project was devised as a means to integrate capacity development within national data analysts' routine workload. Data analysts are situated across all MOH directorates as well as in two of its closely aligned agencies, the Ethiopian Public Health Institute (EPHI) and the Ethiopian Pharmaceutical Supply Agency (EPSA). EPHI and EPSA are tasked with collecting, synthesising, analysing, and presenting data from national RHIS sources to policymakers. The MOH stipulated that participants of the ORCA project should remain full-time employees at their respective institutions, and that activities should avoid interfering with their daily workload. Rather, it was expected that ORCA would play a synergistic role, supporting analysts to develop skills they could apply to their routine work, and encouraging them to overcome vertical working structures and collaborate more closely with counterparts across health agencies. LSHTM established a research office within EPHI in 2015 from which it conducts studies in collaboration with EPHI and MOH staff primarily on maternal and child health service coverage and quality. It was thus well placed to implement the ORCA project continuously over 2 years. The stated aim of ORCA was 'to support and strengthen the capacity of analysts at the Ministry of Health to perform and report high-quality analyses of key health metrics that can inform decision-making'. ORCA brought together staff from the three main Ethiopian health system agencies and across directorates into interdisciplinary working groups arranged around key health themes. This intended to build alliances and collaboration across agencies that often operated vertically with little coordination. ORCA incentivised participation through both individual (professional development grant) and group-based (small operations research budget) financial incentives as a means of increasing participants' motivation and retention during their participation in ORCA, considering the additional work burden incurred [16]. As part of institutional commitment to the project, each participating health agency continued to pay the participating staff's salary and agreed to their attendance in ORCA activities during work hours. ORCA's design drew on theoretical approaches from adult education, which prioritise group-based problem-solving and 'experiential learning', where new skills are introduced in the context of routine work practice [17,18]. These were hypothesised to support a process of normalisation [19], producing work cultures that valued and encouraged routine scrutiny of data quality and efforts to improve it, with the outcomes of higher quality data and more effective analysis and use (Figure 1). We combined residential workshops with small-group coaching, following an operational research cycle to explore extent and reasons for poor data quality within Ethiopia's RHIS. Box 1. summarises all capacity building components included in ORCA. As part of our case study, we adopted mixed methods drawn from process evaluations to prospectively document the intervention. We ensured we could capture fidelity to design (were activities conducted as planned), feasibility of delivery (what challenges were faced and how were they addressed), reach (how many people participated throughout from intended target groups), and acceptability to participants and key stakeholders (how were activities received by those directly involved). We also aimed to capture early signs of intended outputs and outcomes. We did not attempt to measure impact level change, given the short time frame. We used both quantitative and qualitative methods (Table 1). We kept narrative accounts of all ORCA activities, including delays and deviations from the original timing and structure. Routine monitoring data were collated and included project records of recruitment, retention and attendance. An independent researcher (SOA) conducted semi-structured qualitative interviews with eight participants and three managers (one per health agency) 12–15 months into the project. This researcher was not involved in the design or implementation of ORCA other than for qualitative data collection. She was provided with a list of all relevant stakeholders and the 38 participants and purposively selected participants to ensure diversity across sex, role and agencies. All three managers and 7 out of 8 sampled participants were interviewed a second time within 2 months of project completion. The topic guide explored participants' reasons and expectations when joining the project, benefits and challenges experienced during ORCA, and opinions of its usefulness and quality, including suggestions for improvement. Among stakeholders it addressed expectations and experiences, views of the project's overall value, and recommendations. The first round of interviews was conducted in person and lasted 30–60 minutes. Follow-up interviews built on findings from the first interview round, and prob
PDF JSON Files: document_parses/pdf_json/b6534ebde4f61bed3b5ae637957e484c109ce0eb.json
PMC JSON Files: document_parses/pmc_json/PMC8018453.xml.json
G_ID: strengthening_routine_health_data_analysis_in_ethiopia_the_operational_research_and