Annex 6: Monitoring and evaluation of quality abortion care: identification of indicators
Methods and preliminary results
To achieve consensus on indicators that can be used to effectively document progress in quality abortion care at programmatic and country levels, a scoping review and multiple expert consultation processes have been conducted by scientists from the London School of Hygiene and Tropical Medicine (LSHTM) and WHO.
Scoping review: The scoping review yielded a summary of abortion measures and indicators reported over the 10-year period 2008–2018 in scientific literature and materials identified on major non-governmental organization websites. In total, 1999 abstracts and 7 additional relevant documents were screened for indicators, yielding 792 indicators identified from 142 documents. The findings of the scoping review were recently published in BMJ Global Health and the 792 indicators are available on a searchable spreadsheet among the supplementary materials for the publication.
Consultation meeting: At an initial (virtual) consultation meeting in June 2020, multiple sets of indicators that could be used for global- and programme-level monitoring were presented. These included a sub-set of 17 core indicators that had been selected from among the indicators identified in the scoping review and from among indicators for monitoring abortion care based on measurement initiatives implemented variously by the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), the WHO Regional Office for Africa, the Maternal Mortality Initiative (of the WHO Department for Sexual and Reproductive Health and Research) and Ibis Reproductive Health. In total, 52 unique indicators were proposed during the consultation meeting. The WHO health systems strengthening monitoring and evaluation framework was also presented as a potential structure for quality abortion monitoring and evaluation (M&E).
Consolidation: During a consolidation phase, a team of LSHTM and WHO scientists eliminated 29 out of the 52 indicators, removing indicators with duplication of information, using as main review criteria (i) relevance within the health systems strengthening monitoring and evaluation framework, and (ii) feasibility for M&E. Six indicators were added to represent categories in the health system strengthening monitoring and evaluation framework which previously had no associated proposed indicators (such as social protection and infrastructure) and in an effort to align with the Sustainable Development Goals (SDG) targets related to sexual and reproductive health and rights. These six additional indicators were sourced from the original scoping review list of indicators, the Ibis Reproductive Health quality of care indicators, and the WHO Department of maternal, newborn, child and adolescent health (MCA) indicators on respectful care.
Online consultation: In an online consultation in March 2021, measurement and M&E experts in abortion care were invited to comment on (i) the relevance to abortion programmes of the WHO health systems strengthening monitoring and evaluation framework, (ii) the validity, feasibility, importance and sensitivity to change of each proposed indicator, and (iii) suggestions for alternative indicators. Twenty-two (out of 27) invited experts reviewed a total of 29 consolidated indicators during the online consultation. The 29 indicators were categorized in the health systems strengthening monitoring and evaluation framework areas of governance (5), financing (1), infrastructure (1), information system (1), health workforce (1), intervention access and services readiness (5), intervention quality and safety (5), coverage of interventions (1), prevalence of risk behaviours and risk factors (1), improved health outcomes and equity (4), responsiveness (2) and social protection (2).
The core team from LSHTM and WHO is currently finalizing the indicator list following review of input gathered during the online consultation, recognizing that the indicator list will require periodic assessment for strengthening and maintaining utility and relevance. A report will be published in due course.
- Filippi V, Dennis M, Calvert C, Tunçalp Ö, Ganatra B, Kim CR, Ronsmans C. Abortion metrics: a scoping review of abortion measures and indicators. BMJ Global Health. 2021;6(1):e003813. ↩
- Ibid. ↩
- Dennis A, Blanchard K, Bessenaar T. Identifying indicators for quality abortion care: a systematic literature review. J Fam Plann Reprod Health Care. 2017;43(1):7-15. ↩
- Quality of care for maternal and newborn health: a monitoring framework for network countries, updated February 2019. Geneva: World Health Organization; 2019 (https://www.who.int/docs/default-source/mca-documents/qed-quality-of-care-for-maternal-and-newborn-health-a-monitoring-framework-for-network-countries.pdf, accessed 3 September 2021). ↩