Half of global child deaths take place in fragile conflict-affected countries, mainly due to weak health systems with consequent major disruptions of health service delivery. Despite the work of international non-governmental organizations in this type of setting, their role and value in supporting national health systems has not been adequately documented.

KEY FOCUS

The evaluation of the Maternal and Child Health Transformation (MaCHT) project examined the contribution of an international non-governmental organization (INGO) to the efforts of the South Sudan Ministry of Health (MoH) in strengthening the health system’s capacity to deliver essential community-based services in a fragile post-conflict context. The MaCHT project used HBLSS and CCM service delivery packages to train Community Health Workers (CHWs) to provide community-based maternal, neonatal, and child health services.

BACKGROUND & SETTING: WARRAP STATE

Warrap State is located in the northern part of South Sudan and is home to an estimated 1.7 million people. The state is located in flood plains with a long rainy season, and the area is endemic for malaria, and experiences frequent outbreaks of acute watery diarrhea and measles. Within Warrap State, the project covered four of nine payams . The total end-of-project population was estimated to be 148,899.

Health services are delivered through hospitals (at national, state, and county levels), primary health care center (PHCC), and primary health care unit (PHCU). Each PHCC is designed to provide basic emergency obstetric care (BEmOC) services for 25,000 women of reproductive age (WRA) and is staffed by midwives and community midwives. PHCUs are outposts of PHCCs and provide basic preventive and curative series to about 15,000 people.

The project’s target area has seven PHCUs offering basic prevention and promotion services and two PHCCs – both staffed with community midwives and one designated to provide BEmOC.

EVALUATION QUESTIONS

Two main evaluation questions guided the design and implementation to document contributions in improving the capacity of the health system to deliver essential services:

1.To what extent did the project accomplish and/or contribute to targeted goals/objectives?

2.What were the key strategies and factors, including management issues, that contributed to what worked or did not work?

RESEARCH METHODS

The evaluation used a mixed-methods approach, incorporating primary quantitative data with qualitative data.

Quantitative Methods

-808 mothers of children under two were included in the sample; 298 during baseline (FY2011) and 510 during endline (FY2014).

-Participant selection used a two-stage 30 cluster sampling method. Within each selected cluster, households and respondents (mothers of children aged 0-23 months) were randomly selected and only one respondent was selected from a household to avoid overrepresentation.

-A structured knowledge, practice, and coverage (KPC) survey was applied to participants during baseline and endline.

-All collected data was coded and entered into Excel. Data was checked for inaccuracies and inconsistencies, and then entered into SPSS Statistical Analysis software. Data analysis was conducted in two steps. The first step consisted of the production of descriptive statistics for each variable included in the survey. The second included the calculation of p-values (using Mantel-Haenszel two-tailed tests).

Qualitative Methods

In-depth qualitative interviews and focus group discussions were conducted with stakeholders including project staff, MoH, local NGOs and community- and facility-based health workers, community members, community leaders and mothers.

Secondary Data

The evaluator reviewed project reports (e.g. detailed implementation plan; annual reports; mid-term evaluation; baseline and endline KPC surveys; monitoring reports) to assess the quality of quantitative and qualitative data and to assess the project’s results in relation to its design and set targets.

RESULTS

Maternal Care

All seven maternal care-related variables showed significant and positive change over the life of project implementation.

Child Care

All variables related to care-seeking behavior and treatment coverage for diseases measured showed statistically significant increases.

INGOs can have an important role in complementing national health systems’ strengthening efforts. Our findings suggest that community-based models supported by INGOs in fragile-state context effectively and efficiently complement government efforts in the areas of service delivery, human resources (supply and performance of health workforce), information and, with due caveats, supply of medical products.

In South Sudan, community-based health services provided via CHWs and using iCCM are a viable model for increasing MNCH services and practices, and improving access in an overstretched system. Further comparison design research plus additional investments in packaging the model can help prepare it for scale-up.

DISCUSSION

•The project was found to be largely successful in achieving levels of essential health service coverage surpassing the overall basic health service coverage estimated at 40% in South Sudan. According to a Lives Saved Tool (LiST) analysis of six prevention and treatment interventions, there was an overall 7.7% decline on under-5 child mortality over a three-year period (2.6% of annual rate of decline).

•The quality of MCH education was improved in the project area. Over the life of the project, the components of the health system affected positively were human resources (supply and performance), service delivery, and information system.

LIMITATIONS

•Evaluation design:

- Attribution non-feasible

- Over-estimation of results

•Evaluation context: Security, weather, travel distances