Abstract

This paper assesses the Monitoring and Evaluation (M&E) arrangements of the health sector in Zambia. This topic has been pursued with an understanding that sector M&E is an important aspect in the overall national poverty reduction agenda. The specific focus of this paper is on the demand-side of the health sector M&E system. The paper looked at the M&E arrangement in the Ministry of Health (MOH) and the quality of information being generated. For the analytical framework, the paper used the diagnostic checklist elaborated by Bedi et al., in their 2006 publication which identifies four basic elements crucial to consider when assessing the demand-side of an M&E system. These include: (i) analysis and evaluation; (ii) outputs and dissemination; (iii) integration with the budget process and parliament; and (iv) non-state actors as demand-side stakeholders. Fundamental issues brought out are that the M&E function in the health sector in Zambia has some central recognition, but it is not fully developed. It also reveals that the organisational framework to undertake M&E in the sector was in place but equally not yet fully operationalized. Equally, M&E champions exist but not adequate to make an impactful case for M&E. Possible opportunities for a stronger M&E system in the health sector identified are that: Government through the MOH was committed to ensuring that the sector enhances its M&E through various fronts such as donor community involvement through Sector Wide Approaches (SWAps) and the strengthened implementation of the Paris Declaration (PD) principles of alignment and harmonization on aid effectiveness; the link between the National Health Strategic Plan (NHSP) and the budget process through the Mid Term Expenditure Framework (MTEF) mechanism; the presence of the National Decentralization Policy which could be used to strengthen the M&E capacities in the sector especially the lower structures which are currently underdeveloped; the fact that the MOH strategic plan is developed and implemented within the framework of the National Development Plan (NDP)(currently the 7NDP 2017-2021) and coupling this with the use of the MTEF, the arrangements could be used to lobby for resources to strengthen the sector M&E function. The paper also indicates that there is need to create practical incentives for using information generated by the sector M&E system and this could be done in various ways. For instance, linking the use of M&E information to appropriate moments can be effective when well planned and executed. Recommendations for the MOH M&E improvement include; (1) the need to conduct a thorough sector M&E diagnosis; (2) to create a forum for inter-line ministry and stakeholder M&E to share experiences for learning purposes; (3) to build the M&E capacity of lower and decentralized sector structures; (4) to enhance the incentive structure for generating and using M&E information within and outside the MOH; and (5) to engage both MNDP and MOF to provide M&E backstopping.

1. Introduction

Monitoring and Evaluation (M&E) has been considered by many development proponents as a fundamental component in public resource management and is acclaimed for fulfilling accountability, feedback and learning needs of organisations and governments ^{1, 2}. Following the controversial outcomes of the Structural Adjustment Programmes (SAPs) of the 1980’s and early 1990’s, other mechanisms and strategies of development have been elaborated and mostly spearheaded and supported by the donor community. These mechanisms put the developing countries in charge of articulating their development agendas through their own prioritized development plans. These strategies are mostly held in what are called the Poverty Reduction Strategy Papers (PRSPs) or National Development Plans (NDPs). Zambia was among the first countries to develop and adopt the PRSP policy approach and her first PRSP in 2002 was implemented alongside other reforms and reached the Heavily Indebted Poor Country (HIPC) Initiative completion point in 2004 ³. The World Bank and the International Monetary Fund (IMF) guided the country during the PRSP and HIPC processes.

Linked to the PRSP framework, sectoral plans have since been developed in order to involve line ministries and other major development stakeholders in elaborating the national poverty reduction agenda. Through this public policy reform process in Zambia, subsequent strategic plans for the health sector have been elaborated. Planned as a direct input into the NDPs, the Ministry of Health (MOH) developed and is now implementing the third National Health Strategic Plan (NHSP) covering the period 2017 to 2021. The previous strategy was implemented between 2011 and 2015 ⁴. The first NHSP was for the period 2006 to 2010 ⁵. All of these strategic plans have been developed to follow the implementation period ranges of their successive NDPs (FNDP, SNDP & 7NDP).

Like in the NDP, the NHSP has an M&E section that derives its principles from a Results Based Management approach, whose focus is on a management strategy that is aimed at performance as well as achievement of outputs, outcomes and impacts ⁶. The M&E section of the NHSP describes in detail how all the MOH programmes and activities were going to be implemented, monitored and evaluated throughout the life of the sector strategy. In addition, the section elaborates on which specific stakeholders were responsible for what roles and generally how the operational coordination was arranged. In other words, the M&E system is functionally expected to hold and disseminate all the relevant data and information with regards to the MOH programmes and activities.

Thus, in an attempt to bring into context the M&E function within the MOH, the paper provides an overview of the current M&E arrangements in the sector. Ultimately, the paper will argue that when the strengths and weaknesses of what stimulates the demand-side are identified and improved, there could be an upswing in the demand for a better developed sector M&E system. This, in return, becomes a useful motivation for the supply-side to devise ways of satisfying the demand-side of the M&E system.

Before building or strengthening an M&E system, either at national or sector level, it is important to consider as a first step to undertake a thorough diagnosis of the prevailing status in order to map out the strong and weak aspects from both the supply and demand sides of an M&E system. In the literature, different diagnostic instruments have been proposed and elaborated by some interested development oriented stakeholders. A number of M&E proponents have articulated elements that were necessary to consider when dealing with M&E systems. However, it is crucial to note that there is no single agreed upon diagnostic checklist in the literature as being the best of all for use when assessing the quality of M&E systems.

Nonetheless, the available lists are playing a significant guiding role to understanding and appreciating some key components within M&E mechanisms. Suffice to mention, most of these assessment instruments have been elaborated to particularly diagnose national level M&E systems. Nevertheless, the checklists could still be employed and adapted when assessing sector level M&E systems and normally, the sector M&E systems cover a narrow scope as compared to country level M&E systems ⁷.

Among the profound diagnostic tools in the M&E literature include the evaluation capacity building diagnostic guide and action framework ⁸, the readiness assessment ⁹, the diagnostic instrument articulated in the Bedi et al. ¹⁰, the checklist used by Booth and Lucas ¹¹ in their diagnosis of PRSPs and recently the Holvoet and Renard ¹² M&E checklist used in their diagnosis of PRSPs of 11 Sub-Saharan African countries. Of all these elaborations, only checklist that attempts to articulate separately the supply side and the demand side M&E issues is that of Bedi et al ¹⁰. They tried to specify the elements that were critical to consider when looking at either the demand side or the supply side of an M&E system. The other tools are broadly elaborated and discuss both sides simultaneously. For that reason, this paper uses and adapts the Bedi et al checklist to assess the demand-side of the M&E system of the health sector in Zambia.

The components of the checklist include the following: (i) evidence of analysis and evaluation, (ii) dissemination of information, (iii) integration with budget process and parliament, and (iv) participation of non-state actors as demand side actors.

Four objectives guided the study namely: (i) to contextualize the health sector M&E arrangements in Zambia; (ii) to conduct a diagnosis of the demand-side of the M&E system of the health sector; (iii) to discuss possible opportunities for a stronger M&E system for the health sector, and (iv) to recommend actions for improvement.

This paper employed a desk-based-research study and relied mainly on the review of various documents including key Government of Zambia reports (published and unpublished) and other policy related literature. The MOH strategic documents and reports were of significance in this study as well as some scholarly journals, articles and research papers relevant to the topic were consulted.

One notable limitation to this research paper had to do with the inadequacies of disaggregated literature on the supply and demand-side of the M&E systems. Most of the literature available did not state categorically the issues to look for on the demand side. In fact, more attention in the reviewed literature focused on organisational issues of M&E systems, and less on systemic aspects. Nevertheless, the use of different government documents as well as other policy related sources assisted in reducing this problem.

The structure of this paper takes the following order: part two presents a brief literature focusing on the definition of key concepts. Part three contextualizes the M&E arrangements of the health sector in Zambia while part four presents the diagnosis of the demand side of the health sector M&E system. Part five looks at some opportunities that are available to improve the M&E system for the health sector and finally part six gives the concluding remarks and recommendations.

2. Literature

This is a process that is ongoing through which stakeholders obtain feedback on a regular basis regarding the progress that is being made towards meeting their objectives and goals. It broadly involves tracking strategies as well as actions that are taken by partners and non-partners. This also includes figuring out new strategies and actions that have to be taken in order to guarantee progress to the most significant results ¹³. Thus, monitoring is a systematic collection as well as analysis of data for purposes of tracking progress of project or programme implementation against set objectives ¹⁴.

This is an “objective assessment of an ongoing or recently completed project, programme or policy, its design, implementation and results” ¹⁴. The aim of an evaluation is therefore to determine the relevance as well as the fulfillment of goals, the effectiveness and efficiency of development, the impacts and sustainability ¹⁵.

Though different, monitoring and evaluation are complementary in their functioning. “Evaluation is a complement to monitoring in that when a monitoring system sends signals that the efforts are going off track […], then good evaluative information can help clarify the realities and trends noted with the monitoring system” ⁹.

M&E systems refer to “series of policies, practices and processes that enable the systematic and effective collection, analysis and use of monitoring and evaluation information” ^{16, 17}. Additionally, an M&E system broadly includes “the tracking of overall progress in poverty reduction, monitoring and evaluating the implementation of NDP policies and programmes, and the monitoring of budgets and expenditures. The system therefore focuses on the entire results chain (i.e. inputs, activities, outputs, outcomes & impacts) that links the various elements and a broad range of actors” ^{10, 18}.

For any M&E system to be able to provide the needed information, it is important to have both a strong supply-side and demand-side. The supply-side of an M&E system generally refers to the range of systemic and institutional aspects such as data collection, sequencing, leadership, coordination, regulation and oversight. As for the demand-side, it is concerned with the use of M&E information by different actors that include governmental agencies, parliaments, NGOs, CSOs, research institutions, universities, the donor community and the general population. How these entities were involved to stimulate demand for information could be very useful in strengthening the demand-side of an M&E system ¹⁰.

3. Contextualizing M&E Arrangements for the Health Sector in Zambia

In Zambia, M&E has been identified as a critical requirement towards the achievement of sustained economic growth and development. For this reason, M&E has been emphasised and elaborated in all national Poverty Reduction Strategy Papers (PRSPs), now called National Development Plans (NDPs). Similarly, Zambia’s Vision 2030 of becoming a prosperous middle income country also demands for evidence-based development ¹⁹. As Bedi et al ¹⁰ argue, “one of the key components of a successful poverty reduction strategy (PRS) is a system for monitoring and implementation of the strategy and for tracking progress in poverty reduction”. Putting it in another way, PRSPs brought actors together in a participatory manner to agree upon a common poverty reduction agenda and that this was the main theme which M&E systems for PRSPs should be designed to do ¹¹. Thus, M&E systems need to provide relevant information to the places where it will have developmental effects.

Since the implementation of her first PRSP in 2002, the Zambian government has implemented several reforms to improve the performance of the public service, particularly the Public Financial Management (PFM) reforms. Consequently, the pursuit for better M&E has intensified over the years, for example, among many others; Zambia is a signatory to the 2005 Paris Declaration (PD), the 2008 Accra Agenda for Action (AAA) and is implementing the recently launched Sustainable Development Goals (SDGs).

Hence, to be in position of understanding the implementation and resulting effects of the development interventions by government and other stakeholders in Zambia, it is crucial to study and assess the M&E arrangements. More particularly, the M&E systems and arrangements for line ministries are the ones supposed to have detailed information about the poverty reduction efforts in the country. Sectors have comprehensive work plans with mandates to implement the poverty reduction programmes as described in the NDPs, thus, appreciating sector M&E systems offer an opportunity to follow government-wide performance.

At national level, the Ministry of National Development Planning (MNDP) working in collaboration with the Ministry of Finance (MOF) is responsible for the overall coordination of the M&E function in Zambia. There is a separate Division for Development Coordination and Monitoring and Evaluation within MNDP to carry out this national M&E oversight role. The Department of Monitoring and Evaluation under this Division is particularly mandated to coordinate and oversee the evolution of M&E for the entire Zambian public sector ²⁰. Key functions of the department include sector monitoring and developing stronger links between ministries, provinces and districts, including focusing on their capacities to monitor the implementation of their programmes and activities. The department also works with line ministries to develop M&E systems as well as Management Information Systems (MIS). The department conducts regular reporting on performance of the NDP, in consultation with those responsible for monitoring and implementation at the sector level. The production of the Annual Progress Report (APR) on the NDP and the Performance Assessment Framework (PAF) which is used as part of the dialogue process around Poverty Reduction Budget Support (PRBS) is equally the role of the M&E Department. The department also engages district and provincial levels more in the process of monitoring and creating necessary linkages with other stakeholders on such issues as capacity building in areas of data collection, analysis, storage, usage and reporting.

Below is an institutional framework for coordinating, implementing, monitoring and evaluating of Zambia’s Seventh National Development Plan (7NDP 2017-2021).

The institutions and structures in Figure 1 make up the Zambian M&E arrangement linking all NDP functional areas from national to community level. The linkages as depicted show how information is envisaged to flow between the institutional structures.

The MOH has a National Health Strategic Plan (NHSP) with an M&E section that articulates how the plan is being implemented, monitored and will be evaluated. Thus, the health sector M&E system plays a significant contributing role in the Zambian poverty reduction agenda. Therefore, to appreciate the operational nature of the health sector M&E system, it is important to consider looking at the MOH organisational structure. The presence of the M&E function at nearly all levels of the sector is significant to appreciate and this will help in the analysis of the strengths and challenges faced when implementing M&E functions in the sector.

Accordingly, the organizational and management structure of the MOH is a complex one with structures across the country ranging from the national level - all the way through to the provinces, districts up to sub-district or community levels ⁴. What follows is the current institutional arrangement for the health sector and brief functions of each are given with a view to highlight some M&E responsibilities at every level.

At the centre, the MOH Head Office is responsible for the successful implementation of the plan (NHSP) through the formulation and implementation of successive MTEFs, annual action plans and budgets. It is also responsible for policy leadership, management decision-making, standards setting and enforcement, and the overall coordination and implementation of the NHSP. The CAG is the high level consultative and advisory forum for the sector, which brings together MOH and all its partners, including relevant government ministries and departments, private sector, civil society and CPs, to provide advice to MOH on different aspects of health. As part of its mandate, the CAG is responsible for overall steering of the implementation of the plan and monitoring and evaluation of performance. The CAG meets quarterly to review progress, recommend solutions to identified bottlenecks and build consensus on the overall strategic direction of the NHSP.

PHOs serve as intermediaries for implementation, coordination and supporting of the plan (NHSP) within their respective provinces. They represent the ministry’s functional link to the lower level structures, districts, training institutions and the civil society.

District health and hospital management structures are responsible for implementing the NHSP at district and health facility levels. Harmonization of the district and hospital plans to match the aspirations in the NHSP is crucial for successful implementation. DHOs act as links to lower level structures.

Health posts, health centres and hospitals at community level are responsible for the implementation of the NHSP.

These institutions are responsible for the production of appropriately qualified health workers, for implementation of the NHSP.

The following are the key partners for the MOH:

Several other government ministries and departments impact differently on the performance of the health sector, with some actively participating in health service delivery. Strong inter-sector coordination mechanisms have been promoted though currently reported as weak.

The Churches Health Association of Zambia (CHAZ) group is the largest partner to the government in the health sector and is currently the second largest provider of health services to the general public, after MOH. CHAZ plays an important role in the implementation of the NHSP through their network of health facilities, which include hospitals, health centres and health posts, located throughout the country.

In Zambia, the private health sector is not fully developed but is growing. Deliberate efforts are being made to promote private sector participation, including Public Private Partnerships (PPPs), collaboration in research and development, and strengthening of coordination, harmonization and cross-sector referrals ²⁰.

The civil society, both local and international plays an important role in the implementation of the NHSP. Some CSOs are involved in the health promotion, provision of health services, training and capacity building, while others are involved in advocacy for health. MOH works towards promoting stronger coordination and participation of the civil society in the health sector, through the Sector-wide Approach (SWAp) mechanism.

The government works towards strengthening health promotion among the communities and strengthening community involvement and participation in the planning, management, implementation, and monitoring and evaluation of health services, to achieve higher impact. Communities are considered key partners in the success of the NHSP.

The CPs play an important role in the implementation of the NHSP through provision of financial and technical support to the sector and specific programmes. The Government works towards strengthening partnerships with the CPs, and harmonization of their support efforts, for high impact in people’s lives and wellbeing.

It collaborates with the MOH especially through the provision of survey data gathered during the Zambia Demographic and Health Surveys (ZDHS), national censuses and other household surveys.

4. Diagnosis of the Demand-side of the Health Sector M&E Arrangements in Zambia

A thorough diagnosis of the demand-side of an M&E system is a necessary exercise when trying to understand how the utilization of information will potentially benefit the users. Reference ⁹ points out that, “using M&E findings to improve performance is the main purpose of building a results-based M&E system and that information has to get to the appropriate users in a timely fashion so that the performance feedback can be used to better manage organizations and governments”. But to organize M&E information to serve this purpose is not an easy task ²¹. For that reason, the major challenge in M&E is to gather, store and use information that serves different levels ¹³. However, despite this challenge, M&E information which is acceptable is required in order to enhance knowledge that promotes learning. More importantly, M&E information must be disseminated and made available to potential users in order to become applied knowledge.

When M&E systems are developed, and products thereof begin to be distributed, the expectations are that the concerned stakeholders would utilize the information to improve policymaking and better inform management decisions at various levels ⁷. The demand-side of M&E systems is significant because this is the part that consumes the final products distributed after being produced by others (or themselves) from the supply-side. To build and sustain a credible demand-side of an M&E system would therefore be a fundamental effort towards incentivising stakeholders to demand and use M&E information. Hence, as a caution, it must be understood that the demand-side of an M&E system has a way of impacting on the supply-side of the same system ¹⁰. When there is low demand and utilisation of information by key stakeholders, then the M&E function is considered as a mere bureaucratic burden and compliance with the system procedures deteriorates.

In what follows, the paper undertakes an assessment of the health sector M&E system for Zambia and as already stated, the focus is on the system’s demand-side. This diagnostic exercise uses the checklist elaborated by Bedi et. al ¹⁰ which identifies four critical elements to consider when undertaking an assessment of the demand-side of an M&E system. The checklist, as indicated earlier, comprises the following: (i) analysis and evaluation; (ii) outputs and dissemination; (iii) integration with the budget process and parliament; and (iv) non-state actors as demand-side stakeholders. These elements are used as the analytical framework to bring out issues on the demand-side of the health sector M&E system. Using these dimensions of assessment and analysis, the paper strives to show the strengths and weaknesses of the health sector M&E system in Zambia.

One of the most important aspects to consider when building or strengthening the demand side of an M&E system is by ensuring that the practice of analysis and evaluation needs is embedded and well institutionalised within the organisational system. In this way, analysis or evaluation can be beneficial and used for the formulation and justification of policies. Further, it can also build legitimacy and electoral support through evidence of its achievements ¹⁰. Thus, when assessing the health sector M&E system, it is crucial to see how the issue of analysis is being elaborated within the MOH and how seriously it is being undertaken and incorporated.

For the health sector in Zambia, there are challenges regarding the analysis of M&E results and evaluation findings. Although the sector’s main product is the Annual Progress Report (APR), it is filled with inadequacies in analytical content. The APRs were compiled and submitted largely as per donor and central ministries (MNDP & MOF) requirements and less for internal use ²². Since the APR was perceived to be satisfying an external function, there was thus reduced or no incentive at all to invest more in analysis. The sector APRs were submitted to MNDP where the country APR is compiled, but the challenge was that even at national level, no adequate formal arrangement existed to assess the analytical quality of the sector reports ²³. In such instances, it meant that the national APRs equally were a perpetuation of gaps in analytical depth. Consequently, these omissions negatively affected the demand and utility of these reports.

The major problem was the lack of recognition and acknowledgement that the absence of the analysis function was something that required addressing. In the NHSPs, there was silence with regards to the need of M&E data and information analysis. If it existed, then it was taken for granted that the Department of Planning at MOH was the one to undertake the role of analysis. However, the analysis could be best implemented when a separate and independent entity is created. Such a body could be mandated to lead the analysis function on behalf of the sector and situated near the powers that be. The smaller such analysis units remained, the more effective they could be ¹⁰.

However, it was mentioned in the NHSPs that the Cluster Advisory Group (CAG) [formerly known as Sector Advisory Group (SAG)] for health plays the overall function of steering and advising on the operationalisation of the M&E role. To some extent therefore, the CAG is tasked to conduct M&E analytical exercises. Indeed, the creation of a CAG to which health belongs was a positive arrangement but what was not clarified in the MOH documents is the nature of its (CAG) advice and how the incorporation of its recommendations were done to inform not only stakeholders, but the main M&E function in the sector as well.

Further, the MOH conducted Joint Annual Reviews (JARs), a forum that brings together different stakeholders (NGOs, CSOs, other private organisations, etc) concerned with health matters to discuss how health programmes were being implemented in the sector. Strengths and challenges of the sector were reviewed with recommendations for improvement made and these were meant to inform relevant stakeholders to incorporate them for better performance ⁴. Again, there was a limitation in appreciating the outcomes of these reviews because their products were not made public and the APRs too did not give details of the issues that come out, let alone the analytical quality aspects. It is also not clarified how the expertise of other stakeholders were being incorporated in supplementing the MOH M&E capacities.

A sector M&E system that functions well is expected to produce outputs of good quality with information that is credible and useful to the users especially the intended stakeholders. Such an M&E system would normally produce products of high analytical content and compiled into outputs and distributed to actors inside and outside government ¹⁰. At the same time, a functional M&E system will devise a range of outputs appropriate for different audiences and purposes, plus a dissemination strategy that provides those outputs across government and to the public at appropriate moments in the policy cycle. But even more important is how swift the M&E information is disseminated since overtime, the value of information decreases quickly, so important findings have to be communicated as quickly as possible ²⁴.

As indicated earlier, there are many stakeholders within and outside the MOH who were not only concerned with the implementation of sector programmes, but also in using the information for their operations. This means M&E information has to be framed in many but stakeholder specific formats and needs so that demand and usability were simplified and spurred. The M&E system for MOH faced challenges on this aspect. The only notable public document produced and available to stakeholders was the APR chapter, and as indicated earlier it lacked in many ways. Here, the main challenges were at two levels; the first being the incompleteness of the APR itself and second, the weak dissemination strategy of the document (even at national level). Prennushi et al. ²⁵ state that, “M&E needed to be closely linked to decision making processes at all levels and provide feedback to project managers, policymakers, and civil society on, among other things, the performance of existing policies and programmes.” Thus, a crucial element of the M&E system is the existence of a “feedback process or loop”, which is a mechanism by which M&E results were disseminated and used to decide on future courses of action. Simplified products such as reports, press releases, policy briefs, workshops and seminars, newsletters and the Web could be used to disseminate results among the general public and other organisations ²⁵.

Further, APRs lack in addressing the specific information needs of other stakeholders such as the community and other lower structures. In addition, it was also not clear if the timing of the APR production was linked to any appropriate moment in the planning cycle such as the budget process, or even the parliamentary debates. Such links were weak and not clearly mentioned. Thus, silence on linking the APR to appropriate moments or even mere acknowledgement of this necessity undermined the potential to instigate incentives for M&E information demand. Such gaps impacted negatively on the demand-side of the system because MOH will continue to provide less analytical APRs and other reports which actors will not use or demand.

Another important factor on the demand-side involves the budget of the M&E function in the sector. Finances are required to produce, package and to disseminate M&E products that are appropriately stakeholder-oriented and relevant. In MOH, funding to the M&E section was always in deficit and in fact, there was competition with other activities within the Department of Planning and M&E. With the current funding arrangements for M&E activities, very little could be achieved and coupled with high understaffing levels, staff attrition due to low salaries and the generally poor infrastructural capacity in the health sector, a lot may need to be done if the M&E function was to work towards sustainable levels.

Although the sector APRs primarily go to MNDP and there was mention of the MNDP being responsible to provide backstopping services to the health sector M&E, no practical support so far had been acknowledged. Only short and seldom seminars lasting for a day or less and involving all line ministries and other stakeholders were mentioned as having taken place. These seminars were around APR preparations ^{26, 27}. This was not adequate to resolve the many and specific demand-side problems the health sector M&E system was experiencing. Thus, if the analysis and evaluation of products from the health sector M&E system were to be of high analytical quality, the specific sector budget to the M&E function needed to be sufficient and the deliberate involvement of MNDP and MOF would too be of assistance. For now, the loose alliance and involvement of the M&E Department at MNDP and MOF was not making any notable desired positive impact.

At decentralized levels such as the district and community, it was common to find data that was not analysed and packaged into usable reports and formats. Information of this kind could not be used to make decisions at these levels nor could they use such information for lobbying development. To that extent, sector M&E information only benefited the SAG/CAG, the JARs and APRs whose feedback was subsequently shared with the MOF and CPs. Reference ²⁸ in the same line, observes that, “in practice, more attention is usually paid to upward accountability towards the central M&E system and donors as compared to downward accountability towards citizens”. Having a sound upward accountability is good news for Zambia, but there is more need to ensure that there was demand for M&E results at lower levels within the sector as well so that the incentive to even improve on the information quality was enhanced. The absence of public forums to share information limited or hindered citizens’ involvement not only in the implementation of sector plans, but also in M&E activities. The raw data at community level was thus, rendered to be of no use at all

Bedi et al. ^{10, 29, 30, 31} assert that, “creating a link between PRS monitoring and the budget process is a critical objective for a PRS M&E system. The need to access public resources creates powerful incentives across all public agencies and provides the most promising hook for creating demand for effective M&E”. The linkage between the sector M&E system and the budget process is acknowledged as an important element in the Zambian case as well. The health sector operates within the PFM framework as outlined in the NDPs. The Mid Term Expenditure Frameworks (MTEFs)^{1} for the sector were prepared for the period of three years (rolling plan) with annual reviews ³². It was mentioned in the NHSPs that MTEFs were an important tool towards linking the sector plans and the available resources and that this created the basis for M&E function to be able to track the utilization of public finances.

With the MTEFs linked to sector M&E, it was thus expected that high quality analyses would be reflected in the reports, especially APRs which would later be used to inform policy and decision making. In the case of the health sector, it was not clear how the preparation of the MTEFs were influencing the use of the M&E information. Other than being a requirement for upward accountability (mainly to MNDP and donors), there was no notable or recorded evidence that the link between the MTEF preparation and the M&E system had resulted in advantages in improving information quality in the sector. Equally, there was no mention as to whether MTEFs were being developed using information from the sector M&E system. This gap has a negative effect on the demand-side of the sector M&E system because the MOH was still not challenged and obliged to produce quality M&E reports (i.e. no incentive to trigger or spur support to develop both the supply & demand sides of the health sector).

Further, if the MTEF preparation and the disbursement of funds to the health sector was closely tied to the quality of M&E information submitted alongside the budget requests, the motivation to have in place a credible M&E information system would be high. As Kusek and Rist ⁹ affirm that, “with respect to helping formulate and justify budget requests, performance information can inform decisions that can lead to budgetary increases—or reductions. Projects, programmes, and policies may be enhanced or expanded based on performance feedback, likewise, they may be cut or eliminated altogether”. Thus, what seemed to be clear is that the incentive structure in the MOH was lacking or is poorly implemented. In fact, there was no mention of any other form of incentive aimed at hooking the users or the MOH itself towards strengthening the demand-side of the M&E system.

The involvement of parliament in M&E has the potential to bring legitimacy, country ownership, and the voice of constituencies into the process. Among others, some key roles parliament plays include the legislative, oversight, representative and that of reviewing and authorising budgets ^{10, 33}. Therefore, having in place functional linkages between the sector M&E system and parliament could lead to a positive demand and utilisation of system information in critical decisions that impact on poverty reduction planning, expenditures and management.

It is not clearly elaborated how parliament was linked with the M&E function in the Zambian health sector. Further, parliament was not a member of the health SAG/CAG, a platform for various stakeholders with health related mandates meeting to share progress in the sector. This meant that the Zambian parliament only relied on reports (APRs, JARs, etc) produced by the MOH and no deliberate institutional coordination arrangements existed. Since parliament has a health working committee, it was not clear as well what role the committee played in the sector M&E, but most likely this was limited to receiving reports on selected indicators without providing any feedback for holistic improvement of the sector M&E system.

The problem was further compounded by the non-acknowledgement of the importance of the parliament in the sector M&E and the silence could also mean that no institutional arrangements of information flow existed between the two. To that extent, issues of capacity lack in M&E skills at both health sector and parliamentary levels could be hampering the creation of linkages between these different, yet complementary entities. That is why Bedi et. al ¹⁰ argues that, “without a developed committee system, analytical capacity, or sufficient institutional resources, it is difficult for parliaments to engage effectively with the executive on policy issues. Parliamentary capacity was also limited because of a lack of understanding of M&E systems and the opportunities these systems presented for parliamentary engagement”. Despite these shortcomings for the health sector in Zambia, it would have been interesting to know and appreciate how the parliament was involved especially in stimulating the demand for M&E information and how it was applied in improving parliamentary business.

The information produced from M&E systems is primarily meant to be used by various stakeholders, among them governmental and nongovernmental agencies. Other users include the private sector and the donor community. Therefore, care need to be taken to ensure that the information quality is high and wins confidence of these stakeholders. Precisely, the actors play a significant role in demanding for M&E information. Reference ¹² emphasize that, a sector M&E system should be able to indicate explicitly how parliaments, civil society, and development partners use and support the strengthening of M&E systems. So, a strong information demand at all levels is generally the main precondition for the development of a sector M&E system. Sustainable capacity is usually built up if the MOH and other stakeholders are truly committed to measuring the outcomes and impact of public action and to using this information to achieve better results. Thus, the participatory processes followed in designing poverty reduction strategies can be critical in creating a strong demand for monitoring and evaluation ²⁵.

As for CSOs, some of them involved in health service provision were also members of both the SAG/CAG and they took part in JARs as well. This is a positive development for the sector, however, it was again not clear what specific M&E roles these actors provided through their membership and participation. The procedures of civil society participation were not well elaborated and no institutional arrangements existed. Although CSOs were members of other development coordinating committees at national (NDCC), provincial (PDCC), and district (DDCC)^{2}, their particular role in strengthening the M&E function was not articulated in the sector strategy. Such omissions therefore, may have adverse effects on the demand-side of the sector M&E system.

Furthermore, there was also perceived tension between government (MOH) and NGOs to which, Bedi et al ¹⁰ state that, NGOs do not always wish to play an institutionalised role in poverty monitoring for fear that they will be co-opted and become controlled by government. A related problem concerns representation and legitimacy. Whenever CSOs are invited to participate in public agencies and processes, government raises questions as to how representatives of CSOs are selected and whose interests they represented. All these issues compounded the potential development and strengthening of the health sector M&E system.

In addition, the only mention for CPs’ link with sector M&E was their membership and participation in the SAG/CAG and JARs. Thus, to the extent that one of the primary roles of the CAG was to provide overall M&E backstopping to the health sector, the development partners could be said to have some level of institutional arrangement in place. In stressing the need for clear definition of partners’ M&E role, Valadez and Bamberger ²⁴ argue that M&E systems should consider standardizing information needs of donors with those of governments to have uniform M&E procedures. Thus, since the health sector was implementing the PD principles of alignment and harmonization, there may be some agreement among partners on their role in M&E. Some donors were in fact mentioned as being supporting capacity-building efforts in M&E, but all this happened in a less elaborated way as to how exactly this was being done.

Further, the NHSPs also acknowledged the role of the Central Statistical Office (CSO) in the M&E function of the sector. Mainly, the link was with regard to the two evaluations (mid-term and end of strategy) planned during the lifespan of every sector plan, particularly towards the undertaking of the Zambia Demographic and Health Survey (ZDHS) and other household surveys. While this was a positive arrangement, there was no elaboration on how the statistics office helped to strengthen the sector M&E system.

Notwithstanding their usefulness and relevance, there was no mention in the health sector plans how private research institutions and universities in Zambia were interacting to strengthen the demand-side of the M&E system. Surprisingly, the role of the Office of the Auditor General (OAG) was also neither elaborated nor acknowledged in the NHSPs, let alone under the M&E section. Nevertheless, the OAG was responsible for conducting various financial related audits for all government entities and its reports were sent straight to parliament. At that stage, the parliament was able to see what transpired in terms of the financial expenditures of the health sector. But this linkage was not so helpful to the sector M&E system because it was limited to financial checks and not necessarily to the functionality of the sector M&E system. Because of the nature (skewedness to finances) of audit exercises and reports, neither the OAG nor parliament was able to appreciate and influence massive shifts in the functioning of M&E in the sector.

Despite their commitment to the PD’s alignment and harmonization principles, it was not clear what specific role the CPs played in strengthening the sector M&E system. Instead, much was mentioned about the CP’s financial input in other sector programmes such as HIV/AIDS, breast cancer, malaria and so on ⁴. Since they possess different comparative advantages, there were expectations that these stakeholders would consider offering M&E support to the health sector, but this was not acknowledged in the MOH reports either. However, donors could contribute to the creation of demand for improved M&E activities through the requirements of their assistance. For instance, the World Bank and IMF’s conditionality of APRs (in the context of PRSPs) to be presented as requirement for accessing multilateral aid could work as an incentive for the M&E demand-side as well ³⁴. But while such donor requirements did create demand for M&E, sustainable capacity would be built only when there was strong in-country and in-sector demand ²⁵.

5. Opportunities to improve the Demand-Side of the Health Sector M&E System

Results of an M&E system may either be positive or negative and this will always call for transparency and commitment in the manner information is disseminated to the beneficiaries ⁹. Less credible information creates room for stakeholders to speculate and assume results but when the system is transparent and credible, regardless of the quality of information in the reports, stakeholders are bound to abide by the information provided.

Following what has been discussed so far in this paper, a supposition can be drawn to the effect that strengthening the demand-side of the health sector M&E system in Zambia is dependent on many factors. Among the key is the role of stakeholders in demanding to use the M&E information for various decisions at different levels. Although there were many flaws in the current M&E system for the health sector, a number of advantages and opportunities still existed for improvement. The bottom line was for the health sector to continue striving towards building a system that will gradually inspire all stakeholders to use the results for decisions and policy improvements. Therefore, there was need to ensure that enough motivation and demand for improved M&E information was increased and this may only be achieved if the environment was created to allow stakeholders to participate in the MOH programmes. This will call for concerted initiatives on the part of government and on the part of stakeholders themselves. As seen already, the critical setbacks for the sector M&E were around the lack of an independent and sufficient budget, inadequate and uncoordinated analytical capacity, the limitations in information dissemination, and generally the poor incentives for stakeholders to take up active M&E roles ³⁵.

For many years now, the Zambian Government through the MOH has been implementing various reforms aimed at improving the performance of the sector. Some programmes such as the SWAp and basket funds have been used in the ministry for over a decade now. However, their full benefits have not come to fruition yet. Below, the paper presents some existing programmes and practices from within and outside the MOH that may be used to enhance the demand-side of the sector M&E system.

The health sector M&E system does not exist in a vacuum. Instead, it is part of the wider national development strategic framework ³⁶. The NHSPs are linked and developed within the framework of the NDPs. The NDP chapter on health represents a summary of the health sector strategy and thus, the NHSP is an expanded version of the NDP chapter. In the 7NDP, health programmes are articulated under the pillar on Human Development. Thus, upward integration and demand for sector M&E information is established but the only task was to ensure that the M&E linkages and coordination were strengthened especially those between and among MNDP, MOH and MOF. This meant that the MNDP, parliament, CAG and the CPs being part of the demand-side will be motivated to ask for improved M&E results from MOH. The APR preparation and sharing could thus be used to stimulate integration for stakeholders to use as well as for subsequent strengthening of the sector M&E function. Improved analysis of the APR during its preparation stage by all actors could go a long way in creating information credibility, thereby attracting wide usage of the report.

For the M&E demand-side, MTEFs represent a test for the credibility of sector M&E and the MOH generally by developing these frameworks on evidence as reflected in the NHSPs. If the health sector MTEFs are developed following what has been generally agreed upon in the NDPs/NHSPs, stakeholders will be keen to trust and use the M&E information from the sector, knowing very well that interventions were based on mutually agreed upon strategies and indicators. Thus, MTEFs can be used to stimulate demand not only upward but downward as well ³⁷. Therefore, a complete vertical and horizontal integration could be the basis for a sustained M&E demand-side where stakeholders will ask and use the information to improve the management of public resources at various levels.

As indicated by Prennushi et al ²⁵, when a country is undergoing the process of decentralisation of administrative functions and service provision, the most urgent measure would be building M&E capacity at all levels of implementation and management. Sub-national levels such as provinces, district administrations and the communities would need M&E capacities to enable them assess the effectiveness of the strategy pursued at their levels.

Another opportunity for the MOH M&E is the availability of the institutional arrangements made possible through the national decentralisation policy. In 2002, the Zambian Government first launched the National Decentralisation Policy in 2002 and revised in 2013, which aims at empowering the citizenry with an opportunity to exercise control over their affairs and foster development. The National Decentralisation Policy spells out various measures aimed at, among other things, devolving specified functions and authority with matching resources to local authorities at district level ³⁸. Under this environment, the role of the centre (MOH HQ) would be to provide policy, strategic guidelines, overall coordination, monitoring and evaluation while implementation and supervision of the programmes to be done through local authorities at provincial and district levels.

Although the decentralization policy was not fully operationalised, the health sector M&E could already use it as a policy framework to create linkages (vertical and horizontal) so that a network of information flow on health issues in the country is well documented and shared. In practice, the MOH is the most decentralised sector alongside the education sector. This presents a good chance for building strong demand for M&E information in the sector. However, this would only work well after the decentralisation policy was fully or widely implemented especially the budget aspect which was currently still centralised to a larger extent.

Through the SWAp mechanism and the PD on aid effectiveness, the environment and motivation to build a stronger M&E system seems to be available for the health sector in Zambia. Although some donors still had problems to completely align and harmonise their systems with country systems, the overall environment was positive to some extent ³⁹. The Paris Declaration of 2005 and the 2008 Accra Agenda for Action (AAA) spells out the new aid modalities whose demands for donors and aid recipient poor countries were anchored on the five principles of ownership, harmonization, alignment, managing for results and mutual accountability ^{40, 41}. These principles guide the desired reforms which both donors and recipients needed to adopt, adapt and commit themselves to so that there is effectiveness in the overall aid dispensation. Being a big beneficiary of donor assistance, the MOH may too view these changing aid modalities as windows of opportunity to improve the sector’s M&E system.

The call for the alignment and harmonization of donor and recipient country systems would increase the demand for high quality M&E information. The constant demand for highly analytical M&E reports may subsequently lead to a shift within the sector M&E system to improve the credibility of M&E final products.

6. Conclusion

The demand-side of any M&E system is very crucial towards the implementation of a successful and sustainable system. However, the demand-side is not easy to stimulate because it usually involves many actors from both within and outside of any given organisational framework. For example, in the case of the health sector in Zambia, key stakeholders such as parliament, CSOs, NGOs, research institutions the citizens and development partners are found outside of the entity that coordinates M&E. In such instances, it is necessary to put in place deliberate initiatives to inspire and motivate these stakeholders to demand for M&E information. Without certainty in the utilisation of the M&E information, the whole point of having such systems is purely not necessary ^{42, 43, 44}.

This paper aimed at assessing the Zambian health sector M&E arrangements and discuss to what extent the system was developed especially with regard to the demand-side of the M&E system. The sector M&E has been contextualized within the national development framework of the Zambian NDP. An assessment of the current strengths and weaknesses of the sector M&E system has been done using the checklist developed and elaborated by Bedi et. al. ¹⁰. The key elements of assessment included the evidence of analysis, outputs and dissemination, integration, and the role of non-state actors in stimulating the demand-side of the sector M&E system. The diagnosis showed that the Zambian health sector M&E arrangement has a mixed implementation with a combination of successes and failures. Despite this blend, it is crucial to note that the sector has sufficient policy and institutional framework provisions for M&E. What was required is to make full use of the environment especially the issues around the strengthening of the M&E analytical capacity and introducing strong incentives for the use of improved M&E information. The budget constraints for M&E have also been seen as an overarching challenge which needed urgent attention.

Finally, the health sector M&E still has great opportunities for improvement. The paper has shown that there is substantial policy support from the government and indeed the donors. Current global development desires are that governments of poor countries will develop and implement results-oriented frameworks so that the fight against mass poverty for their affected populations is made more feasible and based on evidence. The National Decentralization Policy and other reforms present a window of hope for the enhancement of the sector M&E system.

7. Recommendations

1. Conduct a thorough sector M&E diagnosis: This has not been done before but when undertaken, it will clarify specific points of strengths and weaknesses and this will be a good benchmark to guide all efforts to have a better health sector M&E system. In the absence of this assessment, it is impossible to know the underlying factors for success and failure ⁹.

2. Create forum for inter-line ministry M&E experience sharing: Since government is a broad entity with all line ministries expected to implement the NDP, it will be good for the health sector M&E section to consider sharing experiences with other sectors from different ministries. This will greatly act as a source of knowledge and to help improve M&E practices and capacities ⁴⁵. Other stakeholders may be included ^{46, 47}.

3. Invest in M&E capacity-building at lower structures: This paper has noted that the inadequate engagement of the lower structures such as those at district and community levels had led to poor quality, use and demand for M&E information. However, since the key information for the M&E system is generated at these levels, the health sector should consider creating a sustainable arrangement to bridge this gap. This can prove cheaper for the long term horizon ⁴⁸.

4. Enhance the incentive structure for the M&E function: Innovative incentives need to be introduced to encourage use of performance information. This means that success need to be acknowledged and rewarded, problems need to be addressed, messengers must not be punished, organizational learning is valued, and budget savings are shared ^{36, 49, 50}. This may also mean identifying appropriate moments to anchor these incentives.

5. Engage MNDP and MOF to provide M&E backstopping: The Department of M&E at MNDP has a mandate to help build and strengthen capacities of line ministries in M&E skills and procedures. This is not happening currently although it is acknowledged as crucial. This effort will assist to revolve current inadequacies of M&E issues on the demand-side. MOF need to equally provide deliberate resources towards financing sector M&E.

Notes

1. MTEFs are three-year rolling plans based on the National Health Strategic Plan priorities and strategies, and are considered as important tools for linking the sector planning (i.e. NHSP) and the budgeting process on a yearly basis.

2. NDCC (National Development Coordinating Committee), PDCC (Provincial Development Coordinating Committee) & DDCC (District Development Coordinating Committee). These were advisory and decision-making structures.

References