ciiddaa wmmaallaawii coouunnttrryy pprrooggrraamm … · 2016. 3. 29. · ciiddaa wmmaallaawii...
TRANSCRIPT
CC II DD AA MM aa ll aa ww ii
CC oo uu nn tt rr yy PP rr oo gg rr aa mm
EE vv aa ll uu aa tt ii oo nn 11 99 99 88 -- 22 00 00 88
S y n t h e s i s R e p o r t
Evaluation Directorate
Strategic Policy and Performance Branch
Canadian International Development Agency (CIDA)
J u l y 2 0 1 0
Canadian International Development Agency200 Promenade du PortageGatineau, QuebecK1A 0G4Tel: (819) 997-5006Toll free: 1-800-230-6349Fax: (819) 953-6088(For the hearing and speech impaired only (TDD/TTY): (819) 953-5023Toll free for the hearing and speech impaired only: 1-800-331-5018)E-mail: [email protected]
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
i
Evaluation Directorate – SPPB, July 2010
AA cc kk nn oo ww ll ee dd gg ee mm ee nn tt ss
First, I would like to thank the independent team who undertook the evaluation of the Malawi Country
Program and whose respective responsibilities are indicated below:
Gerry Cooney, Team Leader, responsible for coordinating the evaluation and preparing the Main
Technical Evaluation Report, Governance Sector and Management;
Annette Wenderoth, Consultant, Education and Gender Equality;
Silvia Grandi, Consultant, Health and Gender Equality;
Emmanuel Gaudensio Malangalila, Consultant, HIV/AIDS and SWAp;
Dr. Maxwell Nkhokwe, Consultant, Education.
I would also like to express my gratitude to the Malawi Program Management at CIDA headquarters
(including James Parson, Stefan Paquette and Anne Banwell) and in the field (including Valérie Young,
Roger Roome, and Robert Foote) who, despite their already very busy schedules, took the time to
contribute to this evaluation in many ways by providing documents and participating in interviews, and
for their ongoing support and assistance before, during, and after the data collection missions to Malawi.
I would like to thank all the project managers, technical advisers, and support staff at the Canadian High
Commission and the Program Support Unit who made possible the realization of this Country Program
Evaluation.
Special thanks to CIDA Evaluation Directorate Staff, particularly Maryse Hébert, Senior Evaluation
Manager, who has done excellent work in coordinating this Country Program Evaluation and to Marie-
Eve Fortin and Amélie Geoffroy, who prepared the Program Profile and contributed to the statistics,
tables, and appendices.
Goberdhan Singh
Director General, Evaluation Directorate
Strategic Policy and Performance Branch (SPPB)
Canadian International Development Agency (CIDA)
(Cover photo credit: Silvia Grandi, COMWASH project)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
ii
Evaluation Directorate – SPPB, July 2010
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
iii
Evaluation Directorate – SPPB, July 2010
CC oo nn tt ee nn tt ss
Executive Summary ix
1. Introduction 1
2. Context 2
2.1 Malawi Context 2
2.2 International Development Assistance Context 3
2.3 CIDA Context 5
3. Profile of the Malawi Country Program 7
4. Evaluation Approach and Methodology 10
4.1 Evaluation Questions and Issues 10
4.2 Evaluation Data Gathering 10
4.3 Key Challenges and Limitations 11
5. Sector Overview 12
5.1 Health 12
5.2 Education 12
5.3 Governance 13
5.4 Gender Equality 13
6. Findings by Evaluation Criteria 14
6.1 Introduction 14
6.2 Relevance 14
6.3 Effectiveness/Results 17
6.4 Sustainability 23
6.5 Program and Project Coherence 24
6.6 Efficiency and Resource Management 25
6.7 Principles of Aid Effectiveness 30
6.8 Cross-cutting Issues 31
6.9 Program Management, Structure, and Monitoring and Evaluation 33
6.10 Program Implementation and Delivery Mechanisms 38
7. Conclusions, Lessons Learned, Corporate Considerations and Recommendation 39
7.1 Conclusions 39
7.2 Lessons Learned 40
7.3 Corporate Considerations 40
7.4 Recommendation 41
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
iv
Evaluation Directorate – SPPB, July 2010
EE xx hh ii bb ii tt ss
Exhibit 3.1 Disbursements by Sector and Branch (in $CAD million) 8
Exhibit 6.1 Comparison of O&M Costs to ODA for Selected CIDA Programs
(2006/07 and 2007/08) 27
AA pp pp ee nn dd ii cc ee ss
Appendix I LFA (1999) 42
Appendix II Abbreviated Terms of Reference 44
Appendix III Abbreviated Program Evaluation Matrix and Project Rating Tool 46
Appendix IV Stakeholders Consulted 55
Appendix V Documents Reviewed 62
Appendix VI Projects Reviewed 68
Appendix VII Malawi CPE Project Ratings 70
Appendix VIII Status of the Recommendations of the Malawi Country Program Mid-term
Review (2004) 72
Appendix IX Strengths and Weaknesses of Delivery Models 75
Appendix X List of Findings 78
Appendix XI CIDA's Management Response 81
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
v
Evaluation Directorate – SPPB, July 2010
AA cc rr oo nn yy mm ss
AECPU Aid Effectiveness and Country Program Unit
AfDB African Development Bank
APPR Annual Project Performance Report
BFO Southern and Eastern Africa Directorate
CBS Canada-based Staff
CDPF Country Development Programming Framework
CDSS Community Day Secondary Schools
CEAA Canadian Environmental Assessment Act
CFA Canadian Facilitation Agent/Agency
CHC Canadian High Commission
COMWASH Community Water Supply, Sanitation and Health Project
CPB Canadian Partnership Branch
CPDS Country Program Development Strategy
CPHA Canadian Public Health Association
CPM Country Program Manager
CSO Civil Society Organization
DAGG Development Assistance Group on Gender
DFAIT Department of Foreign Affairs and International Trade
DHRMD Department of Human Resources and Management Development
DPP Democratic Progressive Party
EC European Community
EFA Education for All (Action Plan)
FRAU Financial Risk Assessment Unit
GAP Government Assistance Project
GBV Gender-based Violence
GESP Gender Equality Support Project
GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria
GoC Government of Canada
GoM Government of Malawi
GSES II Grant Support to the Education Sector Phase II
HIPC Heavily Indebted Poor Country
HoA Head of Aid
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
vi
Evaluation Directorate – SPPB, July 2010
AA cc rr oo nn yy mm ss
HoC Head of Cooperation
HoM Head of Mission
HRCC Human Rights Consultative Committee
IDA International Development Association
IMF International Monetary Fund
IPF Indicative Planning Figure
IPS International Policy Statement
IPTE Improving the Quality of Primary Education
IQPE Initial Primary Teacher Education
LAC Latin America and the Caribbean
LDC Least Developed Country
MCO Management and Consular Officer
MCP Malawi Congress Party
MDG Millennium Development Goal
MGDS Malawi Growth and Development Strategy
MGPB Multilateral and Global Programs Branch
MHEN Malawi Health Equity Network
MoEST Ministry of Education, Science and Technology
MoIWD Ministry of Irrigation and Water Development
MPRSP Malawi Poverty Reduction Strategy Paper
NATAFOGAI National Task Force on Gender Advocacy Initiatives
NESP National Education Sector Plan
NGO Non-governmental Organization
NORAD Norwegian Agency for Development Cooperation
OCHA/IRIN UN Office for the Coordination of Humanitarian Affairs /Integrated Regional
Information Network
ODA Official Development Assistance
ODG Office of Democratic Governance
OPC Office of the President and Cabinet
PBO Parliamentary Budget Office
PEG Project for Economic Governance
PIF Policy and Investment Framework
PIP Project Implementation Plan
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
vii
Evaluation Directorate – SPPB, July 2010
AA cc rr oo nn yy mm ss
PMF Performance Measurement Framework
PSU Project Support Unit
RBM Results-based Management
SIDA Swedish International Development Cooperation Agency
SDI Staff Development Institute
SIP Strategic Implementation Plan
SOCAM Society of Chartered Accountants of Malawi
SSTEP Secondary School Teacher Education Project
STD Sexually Transmitted Disease
UDF United Democratic Fund
WATSAN Water and Sanitation
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
viii
Evaluation Directorate – SPPB, July 2010
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
ix
Evaluation Directorate – SPPB, July 2010
EE xx ee cc uu tt ii vv ee SS uu mm mm aa rr yy
Background
Canada has provided development support to Malawi through various programming channels since the
1960s. In 1998, the Canadian International Development Agency (CIDA) approved a 10-year bilateral
country program development strategy (CPDS) aimed at helping Malawi reduce poverty and better
provide for the basic human needs of its people. It focused on two priority sectors: education and health,
and four cross-cutting priorities: HIV/AIDS, gender, governance, and environment. Between 1998 and
2008, CIDA invested CAD$216 million in Malawi through a total of 452 project investments in all
programming channels. The sectors receiving the greatest CIDA support were health (45%), education
(22%), and governance (12%); other sectors received a total of 21% of CIDA disbursements over the
period. In October 2008, CIDA Evaluation Division (ED) contracted Universalia Management Group to
conduct an external evaluation of the Malawi Country Program.
The Country Program Evaluation (CPE) for Malawi was undertaken in response to the Canadian Federal
Accountability Act, which requires all departments to assess the performance of their programs every five
years, and as an input to a Treasury Board requirement to review program-based approaches (PBAs)
including budget support.
This summative evaluation of the Malawi Program had three main objectives: (a) a summative review of
the Malawi Program investments during the 1998-2008 period (taking into account the
results/conclusions/lessons of the mid-term review of the Malawi Program in 2004); (b) an assessment of
the performance of the various delivery mechanisms including program-based approaches (mainly the
Canadian contribution to the Government of Malawi PBA in HIV/AIDS which is currently underway, and
of emerging PBAs including one in basic and primary education, and one in water and sanitation), as well
as directive and responsive projects; and (c) a forward looking analysis of findings and lessons to inform
future programming. However, due to uncertainties about the status of the Malawi Program at the time of
the evaluation, the Malawi CPE was directed more toward accountability purposes than for guidance on
future directions.
Evaluation Methodology
The evaluation team carried out the data gathering and analysis in close consultation with CIDA/ED. The
evaluation approach was consultative, participatory, and utilization-focused, and was executed in
conformity with the principles, standards, and practices set out in the CIDA Evaluation Guide (2004), the
DAC Principles for evaluations (1991), and the DAC Evaluation Quality Standards (2007) which are
consistent with the Treasury Board Evaluation and Policy Standards.
With input from CIDA, the evaluation team developed a detailed methodology and an evaluation
framework. As requested by CIDA, the evaluation team applied a scaled rating tool to quantify
performance information on projects reviewed in detail. The Malawi CPE team made some changes to the
rating criteria in CIDA‟s generic tool to clarify how criteria would be applied at the country program and
project levels. This led to the inclusion of new criteria to: address the cross-cutting issues of governance
and HIV/AIDS, assess the realization of planned/actual outputs and outcomes as part of the effectiveness
criteria, and reduce potential overlap between the coherence and relevance criteria.
Given the timeframe and resources, the evaluation focused on a sample of 18 investments selected during
the work planning stage; these included CIDA bilateral, multilateral, and partnership investments. In
addition, the evaluation team reviewed CIDA‟s support to the PSU over the period in terms of its
contributions to CIDA program management. The initiatives reviewed represented 63% of CIDA‟s
investment during 1998-2008 (approximately CAD$135 million out of a total of CAD$216 million).
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
x
Evaluation Directorate – SPPB, July 2010
There were three major sources of data for this review: people, documents, and site visits. The evaluation
team reviewed numerous documents, consulted 180 stakeholders through semi-structured face-to-face and
telephone interviews, and conducted two site visits to Malawi: a one-week scoping mission during the
inception phase (November 2008) and a two-week data collection mission during the implementation
phase (March 2009). The team used descriptive, content, and comparative analyses to analyze the data for
this study and, to the extent possible, ensured validity through data triangulation (confirming data from
multiple sources). Based on the data analysis, the evaluation team developed findings, conclusions, and
recommendations.
Limitations
Basis for measuring program, sector, and thematic performance – There is no up-to-date
performance measurement framework or tool that provides an appropriate basis for assessing the Malawi
Country Program‟s overall effectiveness. Neither the original program LFA (1999) nor the CPDS
outlined program, sector or thematic level results and related indicators that would have been suitable to
guide the evaluation. The evaluation team focussed on actual achievements and progress towards implicit
results, even if these were not necessarily outlined in the original LFA or the CPDS. In our view, using
the existing LFA or CPDS as the sole basis for assessment would not allow telling the full „performance
story‟ of the Malawi program.
Availability of program-level information – It was difficult to obtain information that captured CIDA‟s
program-level performance in Malawi over the 10-year period due to considerable staff turnover and
documentation that related only to individual projects. In several instances the evaluation team had to rely
on information provided in existing project documents such as progress reports and evaluations, and some
ratings are based on the team‟s interpretation of the available data. Readers should therefore consider the
project ratings as indicative rather than definitive.
Reviewing strengths and weaknesses of delivery mechanisms – The evaluation team assessed the
strengths and weaknesses of the mix of delivery mechanisms used in the Malawi Program. However,
since CIDA‟s portfolio in Malawi included only one example of a program-based approach project (its
contributions to the HIV/AIDS Pooled Fund), the evaluation team could not make a comparison with
other PBAs or make conclusive statements about the strengths and weaknesses of this and other delivery
methods.
Malawi Country Overview
Malawi is one of the poorest countries in the world. In the 2007/08 UNDP Human Development Report it
ranked 164 out of 177. Although the percentage of people living below $1 a day dropped from 53.9
percent in 1998 to 45 percent in 2006, economic growth over the years has not had a significant impact on
poverty levels.1 Malawi is prone to both drought and floods that affect its agriculture-based economy. In
years when agricultural productivity is insufficient to feed the country, or when humanitarian disasters
occur, public and political attention and resources shift to the immediate needs for survival, and social
development issues tend to be sidelined.
Despite these significant development challenges, progress has been made in numerous areas over the
years and the current outlook for Malawi – should present trends continue – is not as bleak as it once was.
Malawi is one of the few countries in Africa on track to meet the MDG to reduce the under-five
mortality rate.
1 Source World Development Indicators database, April 2009. Malawi Data profile. http://ddp-
ext.worldbank.org/ext/ddpreports/ViewSharedReport?&CF=1&REPORT_ID=9147&REQUEST_TYPE=VIEWAD
VANCED&HF=N&WSP=N
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xi
Evaluation Directorate – SPPB, July 2010
Malawi is a top performer regionally regarding sanitation and has made significant progress in
containing HIV/AIDS.
Malawi has been lauded by the world‟s main financial institutions as a responsible spender of
international aid money, and has demonstrated a commitment to achieving sound economic
management and political governance, restoring fiscal discipline and strengthening macro-
economic management. In 2006, Malawi was approved for relief under the Heavily Indebted Poor
Countries (HIPC) program.
The government has launched a large number of parliamentary, civil service, public financial
management, economic and other reforms.
Malawi‟s multi-party political system established in 1994 has brought new freedom of speech and
association. In recent years, political debate has increased, especially among the young and educated.
Civil society organizations (CSOs) have mushroomed since 1994, in particular advocacy and human
rights NGOs. These NGOs, despite serious limitations in terms of capacities and resources, constitute an
important voice in Malawian society today.
Donors provide about 40 percent of Malawi‟s annual budget. Data for 2006/2007 indicated that more than
40 percent of donor spending is targeted towards social development, a field which includes both the
education and health sectors. In the last ten years, CIDA has been the 6th or 7
th largest bilateral donor in
Malawi. However, according to a recent report,2 CIDA was the 15
th largest donor in Malawi in 2007/08
(including bilateral, multilateral, UN agencies, and the Global Fund to Fight AIDS, Tuberculosis and
Malaria [GFATM]). The main donors are: DFID, the World Bank, the EC, the Global Fund, the African
Development Bank, Norway and USAID. China has become a new player in Malawi. In recent years, a
variety of sector specific plans and strategies (e.g., in HIV/AIDS, Health, Education, Water & Sanitation)
have been developed, or are currently being drafted by the Government of Malawi (GoM). Donors are
actively being encouraged to support these joint programs.
Main Findings
Overall, CIDA‟s program in Malawi between 1998 and 2008 was very relevant to the developmental
needs of Malawi. It realized and exceeded most of the developmental objectives as defined in the CPDS.
However, the sustainability of results has varied among projects. CIDA‟s development approach in
Malawi has been generally congruent with the principles of aid effectiveness outlined in the Paris
Declaration. It has realized greatest success at the program level in relation to gender equality as a cross-
cutting issue; its success in other cross-cutting priorities is modest. CIDA has been more successful in
striving for coherence with other development partners than it has been within the Agency and across
delivery channels. CIDA‟s potential efficiency and effectiveness has been limited by the lack of program-
level and sector strategies and monitoring mechanisms as well as its shrinking investment portfolio since
2006.
Relevance – Until 2006, CIDA‟s Malawi Country Program was relevant to the overall development and
poverty reduction priorities of the GoM, to Malawian priorities and needs within the sectors of CIDA
focus. CIDA‟s work has also complemented the work of other donors in these sectors. More specifically:
CIDA‟s contributions to establishing the HIV/AIDS Pooled Fund and strengthening its capacity
have been highly relevant and have led to progress in the health sector. CIDA‟s contribution and
a joint contribution by the Swedish International Development Cooperation Agency (SIDA) and
the Norwegian Agency for Development Cooperation (NORAD) tied as 3rd
out of nine donors to
the pooled fund.
2 Malawi Aid Atlas (FY 2007/08) GOM, 2009. Information on CIDA‟s de facto contributions to Malawi in 2007/08
are understated in the Malawi Atlas due to the timing of CIDA‟s disbursement to the HIV/AIDS Pooled Fund which
was made in March 2007 prior to the GOM‟s 2007/08 reporting period.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xii
Evaluation Directorate – SPPB, July 2010
Similarly, CIDA‟s strategic support in the water sector has been congruent with GoM plans to
design a Water and Sanitation SWAp. In 2007/08, Canada ranked 7th out of eight donors in
Malawi that supported the water sector.
In the education sector, CIDA programming has been aligned with and has addressed some of the
key priorities outlined in the 2001 Malawi Policy and Investment Framework (PIF) for the
education sector as well as in the 2008 National Education Sector Plan (NESP). In 2007/08,
Canada ranked 9th among Malawi‟s 13 donors in the education sector.
Canada was one of a very few donors to support improved governance in Malawi a decade ago,
with a particular focus on strengthening the National Assembly, the Public Service, and the role
played by civil society in Malawi. Other donors, including DFID, are now active players in the
sector. In 2007/08, CIDA ranked 9th
among Malawi‟s 11 donors in the governance sector.
CIDA‟s focus on gender equality-related capacity building, awareness-raising, and technical
assistance were aligned with Malawi National Gender Policies (2000 and 2008) and the Malawi
National Gender Program (2005). In 2007/08, Canada ranked 5th out of Malawi‟s eight donors
that supported gender equality.
Canada‟s relevance decreased slightly in recent years due to shifts in GoM priorities from social
development to economic development and infrastructure.
Effectiveness – Most individual investments reviewed met their planned outputs and often their
outcomes. The Malawi Program has made considerable achievements, in particular in health (HIV/AIDS
and water sub-sectors), education, governance and gender equality as illustrated below:
HIV/AIDS: CIDA made significant contributions to progress in the HIV/AIDS sub-sector,
mostly through its involvement in the HIV/AIDS Pooled Fund, the first program-based approach
established in Malawi. The support of pooled fund partners contributed to the capacity and
effectiveness of the National AIDS Commission (NAC) in coordinating Malawi‟s national
response to HIV/AIDS. This has contributed to a significant increase in the antiretroviral therapy
coverage in Malawi, with a special focus on pregnant women, an increase in the number of HIV
testing and counselling sites, and improved support for orphans and vulnerable children among
other results.
Water and Sanitation: CIDA support to the Ministry of Irrigation and Water Development
(MoIWD) helped it to pilot community-managed water schemes, contributed to GoM policies and
practices for water and sanitation (e.g., National Water Development Program, Malawi Sanitation
Policy), played an instrumental role in fostering donor coordination through the Water and
Sanitation Development Partners group, and contributed to planning for a Water and Sanitation
SWAp.
Education: CIDA has made visible and significant contributions to progress in the education
sector, particularly in the areas of capacity building for textbook supply and procurement, and to
a lesser degree, in secondary teacher education.
Governance: CIDA support helped to increase civil society and public participation in national
budget processes in Malawi and thus increase their influence on budget priorities. It also helped
to increase the capacity of Parliamentary Committees to scrutinize issues relevant to poverty and
social protection, provide oversight on government spending, and conduct public hearings.
Gender equality: CIDA is widely recognized for its concrete achievements in gender equality in
Malawi and for its role in continuously encouraging its partners to address gender in their own
work. According to consulted donors, NGOs, and GOM representatives, CIDA‟s work on gender
has increased its visibility and reputation in Malawi.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xiii
Evaluation Directorate – SPPB, July 2010
While there is considerable evidence that the program achieved results in all of CIDA‟s priority sectors
in Malawi, it is difficult to assess effectiveness at sector and program levels due to the absence of sector
and program level strategies, expected results and indicators, and a system to track such results.
The CIDA Malawi Program established a positive Canadian reputation among a broad range of
stakeholders in Malawi. It was very successful in leveraging resources of other donors to support
Malawi‟s development priorities. However, the long period of uncertainty and the recent decisions about
Canada‟s future presence and level of engagement in Malawi have negatively affected Canada‟s
reputation.
Sustainability – While CIDA support has contributed to an impressive number of results in Malawi since
1998, the likelihood of these being sustained is quite mixed and modest overall. This reflects Malawi‟s
challenging context as well as some of CIDA‟s practices in managing for results.
Some good practices that are likely to help support the sustainability of results include:
Institutionalization of new approaches, programs and policies in Malawian institutions (e.g.,
MoEST procurement unit, distance education for secondary teacher training, readmission of teen
mothers to school, Bill and Plan of Action on Prevention of Domestic Violence).
Diversifying funding sources (e.g., the HIV/AIDS Pooled Fund) and attracting funding from other
agencies (e.g., CIDA investments in civil society, media, Parliament, and Ministry of Finance
attracted funding from UNDP, DFID, Trocaire, Oxfam, Irish Aid, EU, and others).
Some practices (or absence thereof) were less successful in supporting sustainability, including:
Absence of sustainability strategies at both program and project levels (e.g., in many reviewed
projects, sustainability was not considered until the project was closing)
Short-term investments and lack of follow-up (e.g., while short-term initiatives may be
appropriate in some cases, given the limited absorptive capacities of organizations, many of
CIDA‟s catalytic initiatives are not likely to be developed or sustained).
Coherence – While country program managers for Malawi made some efforts to build coherence across
bilateral projects in Malawi within and among the program‟s themes, connections among such projects
have been largely accidental. CIDA has made some efforts to link bilateral priorities and Canadian
Partnership Branch (CPB) and Multilateral and Global Program Branch (MGPB) projects. This reflects
the fact that the CPDS was a not a truly corporate document that was binding on non-bilateral channels
and that coherence among CIDA branch programs was not an over-riding CIDA priority during the period
under review.
Efficiency and Resources Management
Project level efficiency: Most bilateral projects were completed within budget and on time, and
some leveraged funding from other donors – e.g., Project for Economic Governance (PEG) and
the Gender Equality Support Project (GESP). The bilateral projects reviewed had a mix of
efficiency ratings, from highly satisfactory to unsatisfactory. Unfortunately, evaluations of many
CIDA projects in Malawi did not assess project efficiency and the CPE team could not rate these
projects in terms of their efficiency.
Program level efficiency: At the country program level, a review of the ratio of operations and
management (O&M) expenses to ODA for the Malawi program indicates that its management
costs were relatively modest in 2007 and 20083 (6.5% and 8.5%) respectively, and similar to
3 Due to changes in CIDA financial systems, the CPE was not able to obtain comparable data for earlier years and
focused analysis on 2007 and 2008.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xiv
Evaluation Directorate – SPPB, July 2010
other similar CIDA programs. While CIDA uses the ratio of O&M to ODA expenses as a proxy
measure for assessing program efficiency, it excludes some management costs, such as PSU
expenses, which are currently considered by CIDA as developmental rather than O&M costs.
Resources management (Program management): Despite the CPDS‟s stated emphasis on
sectors and themes, most bilateral resources were allocated to project needs, which limited the
potential for strategic program-level management, synergy, and learning. Until 2004, the Malawi
Program managers focused on collaborating with partners to grow the program. Since 2006, the
managers have concentrated on managing existing investments and generating information to
determine the program‟s future. The absence of a clear formal arrangement between CIDA and
DFAIT regarding the common services that would be provided to CIDA to support its
programming in Malawi contributed to several program inefficiencies.
Resources management (PSU): The PSU in Malawi greatly facilitated the start up of the
program. Over the past decade, its technical expertise in education, HIV/AIDS, water, and gender
equality has been highly respected and appreciated by the Malawian authorities and the donor
community. However, as a consequence of the program‟s uncertain status since 2006, the PSU
had limited opportunity to assume a role in program development and analysis as envisaged.
Principles of Aid effectiveness – At the project level, CIDA‟s development approach in Malawi has been
generally congruent with the principles of aid effectiveness outlined in the Paris Declaration as illustrated
below:
Ownership: CIDA actively sought GoM ownership in several ways during the design of
individual projects. There was a strong focus on capacity development of GoM departments and
entities in several projects such as the Grant Support to the Education Sector Phase II (GSES II),
the Community Water Supply, Sanitation and Health Project (COMWASH), and the HIV/AIDS
support program. CIDA supported mechanisms such as SWAPs and pooled fund arrangements
that could be led by the GoM. In other instances, CIDA encouraged the active involvement of
Malawi government ministries in project oversight.
Alignment: The evidence suggests that the CPDS was aligned to the priorities identified by the
GoM in its main development strategies. The CIDA Malawi Program and individual projects
were designed on the basis of reviews of existing national strategies, priorities, and needs, using
extensive consultations with GoM and other local and international stakeholders.
Harmonization: CIDA played an active and acknowledged role in donor coordination and
harmonization in Malawi, notably in the HIV/AIDS Pooled Fund and in the preparation for other
possible SWAps in water and sanitation and education. It also played an active role in developing
partner sector coordination groups and, in several cases, provided the Secretariat for these groups.
CIDA‟s positive record in donor coordination and harmonization has been weakened in recent
years by its limited financial commitments for SWAps.
Results: All CIDA partners in Malawi acknowledge the Agency‟s focus on results, and credited
it for having supported RBM capacity building among its partners for example through the GESP
and within the Development Assistance Group on Gender (DAGG). However, CIDA has been
more effective at utilizing these approaches at the project level, rather than sector, thematic or
program levels.
Mutual accountability: At the project level, CIDA has shared project reports and evaluations
with relevant ministries and other key stakeholders. While CIDA did report regularly to the GoM
on financial expenditures in Malawi, it is not evident that CIDA informed the GoM of
developmental results at the program, sector, or thematic levels on a formal, regular basis.
CIDA‟s limited investments in Malawi after 2006 combined with CIDA‟s limited consultations
and communications to determine the future of the program in Malawi raised concerns by its
partners about its future assistance plans. Lessons from other donors highlight the importance of
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xv
Evaluation Directorate – SPPB, July 2010
open dialogue and a respectful process that recognizes mutual accountabilities. Following the
recent decision on geographic focus, the Malawi Program has been having discussions with the
GoM regarding programming plans.
Cross-cutting issues – Consulted stakeholders commended CIDA for its attention and commitment to
gender mainstreaming, and there is evidence that CIDA-supported projects made significant contributions
in this area. At the program level, it is difficult to track or assess the program‟s effectiveness in
mainstreaming these issues as the program did not establish specific objectives, strategies, and practices
or allocate resources and responsibilities to systematically address cross-cutting issues. At the project
level, CIDA management paid attention to cross-cutting issues (gender, HIV/AIDS, governance, and
environment), particularly to gender equality. Environmental concerns were not an issue in the type of
projects funded by CIDA, but all projects complied with the Canadian Environmental Assessment Act
(CEAA) principle to “do no harm”.
Program Management, Structure, and Monitoring & Evaluation
Strategic management: To launch a country program in Malawi in 1997-1998, CIDA crafted a
focused, distinct, innovative, and bold development strategy that was carefully designed to
maximize the impact of Canada‟s relatively modest resources. The CPDS for Malawi guided
CIDA bilateral investment decisions in Malawi up to 2004. Since then it has had much less
influence over the program‟s investment decisions and directions, primarily due to the uncertainty
surrounding the future of CIDA‟s bilateral programming in Malawi. The program‟s LFA had
limited utility as a basis for capturing and measuring performance beyond the project level. In
2004, CIDA conducted a Mid-Term Review of the Malawi Program to review results
achievement, sustainability of results, relevance of the program and its effectiveness. The Mid-
Term Review made nine recommendations. An analysis of the status of these recommendations
(see Appendix VIII) indicates that seven of the nine recommendations were partially, mostly, or
fully addressed. The recommendation to create a new position in the Malawi program was not
addressed, largely due to the uncertain status of the program. The recommendation that the
program establish and maintain an appropriate program LFA was marginally addressed: the LFA
was updated but was never formally approved.
Structure and leadership: The CIDA Malawi Program‟s atypical management structure
between 1998-2003 (in which the PSU Director filled the role of Head of Cooperation in addition
to his formal role) allowed the program to become operational quickly. The combination of
strong leadership, close collaboration with development partners in Malawi, staff continuity and
dedication, and flexible procedures helped Canada earn a very positive profile and reputation in
Malawi until 2004. A combination of staff turnover and a vacuum in CIDA leadership in 2004/05
negatively affected the performance of CIDA‟s Malawi Program. Starting in 2006, the
uncertainty about the future of the program continued to affect its management and leadership.
RBM and M&E: Over the past decade, the program concentrated mainly on project-level
monitoring and reporting. In all cases, projects have applied RBM principles and tools in
meaningful ways and have used appropriate project management systems and resources, as well
as risk identification and mitigation processes. At least one project developed their partners‟
capacities in RBM. CIDA had various mechanisms in place, including the use of external
monitors, to identify risks at the project level. Similar mechanisms did not exist at the program
level.
Delivery and funding mechanisms – The Malawi and CIDA contexts over the decade warranted
CIDA‟s use of a mix of program implementation and delivery mechanisms, such as project-based and
program-based approaches (i.e., SWAps, pooled funding). Most consulted stakeholders confirmed the
validity of CIDA‟s approach which mixed different delivery and funding mechanisms and spread
investment risks.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xvi
Evaluation Directorate – SPPB, July 2010
Conclusions
The findings of this evaluation need to be understood within the context of the Malawi Program's
evolution over the past decade. The 10-year CIDA Malawi Country Program had three distinct phases.
The first phase (1998-2003) was characterized by the excitement of establishing a new program and
relationships. The period was marked by a focused CPDS, strong Canadian leadership, and close
cooperation with development partners in Malawi. The second phase (2004 and 2005) was marked by
major changes and transition in the program‟s management structure, high turnover of several individuals
in leadership positions in Canada and Malawi, and changes in relationships between CIDA and its
partners in Malawi which had marked effects on the program‟s performance. In the third phase (since
2006), the uncertain status of the bilateral program in Malawi within Canada‟s overall aid program,
modest program investments, and continued personnel turnover had a negative effect on those working in
the program and on relations with Canada‟s partners in Malawi.
Given the challenges and constraints it faced, the Malawi Country Program has performed well in terms
of the results achieved and the relevance of its financial investments and technical expertise in health,
education, governance, and gender equality. It gained considerable respect from the Malawi government
and other donors and partners for its leadership in gender equality and HIV/AIDS and its role in
encouraging and supporting donor coordination. Its shortcomings relate to the lack of program-level and
sector strategies and monitoring mechanisms, the sustainability of some project results, and the efficiency
of the program in recent years.
Lessons Learned
Concerning country program design and management:
To be effectively managed, programs require deliberate strategies, accountabilities, mechanisms,
and resources for planning, monitoring, and reporting at both program and sector levels.
Concerning managing for sustainable results:
The sustainability of project results is more likely when sustainability is addressed at the project
design stage and considered throughout implementation, not in the last year(s) of a project.
Project managers are more likely to pay continued attention to sustainability of results if this is
identified and managed as a priority in project and program management practices.
Concerning program transition:
Early warning and clear communication with key stakeholders are necessary ingredients in
respecting good partnership principles and fostering a smooth transition when there is a major
change in program status.
Concerning CPE design and management:
A CPE can assess only those dimensions and concepts in a meaningful way that have actually
been relevant for and have been used by (or could have been used by) the respective program
(e.g., the concepts of program coherence and efficiency).
Carrying out several CPEs at the same time can create opportunities for synergy, as it is likely
that all evaluations will be struggling with similar issues. Opportunities for synergy can be lost
however if: i) scheduling does not allow time for systematic exchange between evaluation teams,
and ii) commonalities between different CPEs are merely assumed in prescriptive methodological
guidelines, but are not backed up by empirical evidence or actual experiences gained during the
evaluation process.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
xvii
Evaluation Directorate – SPPB, July 2010
A CPE is most valuable when conducted at a point in time where the findings and
recommendations can actually be used to inform planning and decision-making regarding the
program‟s future.
Corporate Considerations
The overall good achievements of the Malawi Program notwithstanding, the evaluation identified areas
where improvements can be made to enhance this performance. Some of these pertain to broader Agency-
level improvements, such as the issue of whole-of-Agency approach, the strengthening of performance
management processes and the guidance on managing country program transitions. Considering that the
Agency has taken decisions on these aspects, and actions are underway to address them, as indicated in
CIDA‟s Management Response to the Office of the Auditor General of Canada (OAG) and in CIDA‟s
Aid Effectiveness Action Plan, they will not be the subject of specific recommendations.
Recommendation
Given uncertainties about the status of CIDA‟s future program in Malawi at the time of the evaluation, the
TORs for the Malawi CPE excluded an analysis of CIDA future programming directions in the country.
For this reason, the Evaluation Team does not provide any recommendations to the Malawi Program
related to its future strategic direction.
One recommendation is addressed to the program on management issues. – The Malawi Program remains
in transition from a core program to one of modest presence. It is generally expected that the program
budget will be much smaller than before, with a smaller program infrastructure in Canada and abroad.
Recommendation 1: The managers of the CIDA Malawi Program should identify efficient ways to
deliver this modest presence country program.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
1
Evaluation Directorate – SPPB, July 2010
11 .. II nn tt rr oo dd uu cc tt ii oo nn
Canada has provided development support to Malawi through various programming channels since the
1960s. In 1998, the Canadian International Development Agency (CIDA) approved a 10-year bilateral
country program development strategy (CPDS) aimed at helping Malawi reduce poverty and better
provide for the basic human needs of its people. It focused on two priority sectors: education and health,
and four cross-cutting priorities: HIV/AIDS, gender, governance, and environment. Between 1998 and
2008, CIDA invested CAD$216 million in Malawi through a total of 452 project investments in all
programming channels. The sectors receiving the greatest CIDA support were health (45%), education
(22%), and governance (12%); other sectors received a total of 21% of CIDA disbursements over the
period.
CIDA‟s Performance Review Policy calls for periodic, independent evaluations of the Agency‟s
investments to assess results, inform decision making, and promote organizational learning,
accountability and transparency within the Agency. The Malawi Country Program Evaluation (CPE) has
been undertaken as a response to the Canadian Federal Accountability Act, which requires all
Departments to assess the performance of their program every five years. Further, when approving the
Terms and Conditions for CIDA in 2007, the Treasury Board asked it to perform a review of its program-
based approaches (PBAs), including direct budget support.
In October 2008, the CIDA Evaluation Division (ED) contracted Universalia Management Group to
conduct an external evaluation of its Malawi Country Program. The evaluation had three major
objectives:
1) A summative review of the Malawi Country Program investments during the 1998-2008 period
(taking into account the results/conclusions/lessons of the mid-term review of the Malawi
Program in 2004);
2) An assessment of the performance of the various delivery mechanisms including program-based
approaches (mainly the Canadian contribution to the Government of Malawi PBA in HIV/AIDS
and of emerging PBAs including one in basic education, and one in water and sanitation), as well
as directive and responsive projects; and
3) A forward looking analysis of findings and lessons to inform future programming.4
This summative evaluation of the Malawi Country Program covers the implementation period of the
CPDS from 1998/99 to 2007/08, including all disbursements made up to March 31, 2008. Given the
nature and the scope of the resources, the evaluation focused primarily on the bilateral program, but also
considered the investments of other branches. The primary audiences for the evaluation are CIDA‟s
Strategic Policy and Performance Branch (SPPB) and CIDA Malawi program staff at HQ and in the field.
Secondary audiences include CIDA Southern and Eastern Africa Directorate, Geographic Programs
Branch (BFO/GPB), Canadian Partnership Branch (CPB), Multilateral and Global Programs Branch
(MGPB), as well as program partners in Canada and Malawi, in particular the Government of Malawi
(GoM).
This synthesis report outlines the main conclusions and findings of the evaluation. A list of findings is
presented in Appendix X. The full technical report is available from CIDA upon request.
4 Given uncertainties about the status of CIDA‟s future program in Malawi at the time of the evaluation, the TORs
for the Malawi CPE excluded an analysis of CIDA future programming directions in the country. For this reason,
the Evaluation Team did not provide any recommendations to the Malawi Program related to its future planning and
programming. This approach has varied from other CPEs for which looking forward was an important objective to
guide their review.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
2
Evaluation Directorate – SPPB, July 2010
22 .. CC oo nn tt ee xx tt
22 .. 11 MM aa ll aa ww ii CC oo nn tt ee xx tt
Overall – Malawi is one of the poorest countries in the world and ranked 164 out of 177 countries in the
2008 UNDP Human Development Report. Although the percentage of people living below $1 a day
decreased about 9% in the last ten years, economic growth over the years has not had a significant impact
on poverty levels. Malawi is prone to both drought and floods that affect its agriculture-based economy.
In years when agricultural productivity is insufficient to feed the country, or when humanitarian disasters
occur, public and political attention and resources shift to the immediate needs for survival, and social
development issues tend to be sidelined.
Health data in Malawi depict a relatively bleak situation: Malawi has one of the world‟s highest maternal
mortality ratios, HIV prevalence remains high, with severe consequences for the country‟s public health
system and the overall productivity and social fabric of the nation. In education, while enrolment rates in
primary education are high (94 percent for girls and 88 percent for boys in 2006), only 55 percent of
children complete a full course of primary education. Malawi‟s literacy rate is higher than the regional
average (approximately 71 percent of adults and 82 percent of youth).
Despite significant challenges, Malawi has made progress in numerous areas over the years. It is on track
to meet the MDG to reduce the under-five mortality rate, and has surpassed its 2015 targets for drinking
water. It is a top performer regionally regarding sanitation and has made significant progress in containing
HIV/AIDS. Malawi has been lauded by the world‟s main financial institutions as a responsible spender of
international aid money, and has demonstrated a commitment to achieving sound economic management
and political governance.
Political Context – Since its peaceful transition to multi-party democracy in 1994, Malawi has held three
elections. In 1999, Bakili Muluzi of the United Democratic Front (UDF) was re-elected as President for a
second term. His successor, economist Bingu Wa Mutharika was elected in 2004 and re-elected in 2009
and his party, the Democratic Progressive Party (DPP), won a strong parliamentary majority. President
Mutharika launched a crackdown on corruption and took steps to improve fiscal discipline, actions which
restored the confidence and support of international donors.
During the past decade, the government has launched a number of reforms (parliamentary, civil service,
public financial management, economic, etc.) and national level plans and strategies: Malawi Vision 2020
(2000); the 2002 Malawi Poverty Reduction Strategy (MPRS); the 2003 Malawi Economic Growth
Strategy (MEGS); the 2006 Malawi Growth and Development Strategy (MGDS) 2006-2011; and the
2006 Malawi Development Assistance Strategy (DAS) 2006-2011.
The multi-party political system has brought new freedom of speech and association. Political debate has
increased, especially among the young and educated, and civil society organizations have mushroomed
since 1994, in particular advocacy and human rights NGOs.
Cultural Context – During the Banda era the main values in Malawian society were unity, obedience,
discipline and loyalty. While the public rhetoric increasingly moved away from the paternalistic and
authoritarian values that dominated public discourse for generations, these values still influence large
portions of Malawian society. Traditional rules and behaviours prevail, especially among the rural
population, and continue to influence social norms, expectations, and behaviours.
Unequal gender roles are deeply rooted, and male authority over women is widely accepted. On the
UNDP gender equality-related development index Malawi ranks 139 out of 157 states. Literacy among
adult women (63 percent) is lower than among male counterparts (79 percent). While girls have high
enrolment rates in primary school, they have a much higher drop-out rate than boys, mainly due to
poverty, early motherhood, and care responsibility for chronically sick family members (due to AIDS,
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
3
Evaluation Directorate – SPPB, July 2010
TB or other illnesses). Gender equality in secondary school is far from being attained: in 2007, the ratio of
girls to boys in secondary schools was 0.70 to 1. Women‟s share in paid employment in the non-
agricultural sector increased from 13 percent in 2000 to 15 percent in 2006. Women‟s representation in
Parliament increased from 9 percent of seats in 1999 to 21 percent in 2009, largely due to a civil society
campaign. In Cabinet, women‟s
representation is 26% including the
Vice-President.
Development Aid in Malawi –
Donors provide about 40 percent of
Malawi‟s annual budget. According
to the GoM Malawi Aid Atlas (FY
2007/08), the main donors are: DFID, the World Bank, the EC, the Global Fund, the African
Development Bank, Norway and USAID. In the last few years, China has become a new player in
Malawi. Some observers are concerned that China‟s unconditional aid and low interest loan strategy may
undermine long-term efforts to build a democratic and accountable administration in Malawi.
In 2006/2007, more than 40 percent of donor spending was targeted to social development (which
includes the education and health sectors). The GoM raised concerns in the Joint Country Program
Review in May 2007 that other key areas such as economic growth and infrastructure are not being given
sufficient attention.
Over the past decade, the GoM has shown increasingly active leadership with regard to development
strategies, aid, and donor coordination. It is committed to the principles of aid effectiveness as outlined in
the Paris Declaration and ranks in the top ten countries globally in terms of aid effectiveness.
The GoM is working hard to achieve the Paris Declaration objectives on aid effectiveness. A strong unit
in the Ministry of Finance monitors all aid to Malawi and encourages donors to work within GoM
priorities. The GoM suffers from human resource and institutional weaknesses, but donors have noted
progress in these areas. Donors themselves must make further efforts to align with national systems,
harmonize procedures, consult with the GoM, and provide aid in a predictable manner.
The 2006 Malawi Development Assistance Strategy (DAS) states that the Ministry of Economic Planning
and Development should be involved in the planning of all projects that development partners are asked
to fund to ensure that they align with the MGDS. It further states that “the government‟s preferred
modalities of aid delivery are budget support and basket funding within SWAps.” Development partners
are encouraged to provide up to 70 percent of their development assistance using these modalities. Donors
are increasingly harmonized in the provision of development assistance. Program-based approaches
(PBAs) are in effect in health and HIV/AIDS, and new PBAs in education and water are nearing
completion. In addition, the United Kingdom, Norway, the European Union, the African Development
Bank, and the World Bank provide general budget support. To date, however, much of the aid delivered
by the international community still takes the form of project support. In 2006/2007, project support
amounted to two-thirds of total aid disbursed.
22 .. 22 II nn tt ee rr nn aa tt ii oo nn aa ll DD ee vv ee ll oo pp mm ee nn tt AA ss ss ii ss tt aa nn cc ee CC oo nn tt ee xx tt
International events and changes in aid architecture in the past decade have had major impacts on
relationships between international development agencies and their development partners. Increased
global attention to and investment in peace and security concerns, as well as the global financial crisis, are
affecting priorities for foreign aid and will likely have adverse effects on developing countries such as
Malawi that are seen as having limited strategic importance.
Global declarations – The ways in which development aid is delivered have been influenced by global
events and declarations such as the Millennium Development Goals (2000), the UN Monterey summit
(2002), and the Paris Declaration on Aid Effectiveness (2005). The Paris Declaration principles have led
Canada’s development support to Malawi
CIDA was the 15th largest donor in Malawi in 2007/08 (including
bilateral, multilateral, UN agencies and the Global Fund). In the last ten years, CIDA has been the 6
th or 7
th largest bilateral donor in
Malawi.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
4
Evaluation Directorate – SPPB, July 2010
to a stronger focus on managing by and reporting on results, and to a greater emphasis on supporting
national development plans. There is an increased focus on delivery mechanisms that maximize country
ownership while aligning and harmonizing donor efforts, including PBAs and budget support. Good
practices and lessons learned are emerging, but donors and partner countries are still experimenting.
While the Paris Declaration has become a reference point for most bilateral and multilateral donors, there
has also been growing critique of its gaps and shortfalls. One common criticism has been its lack of focus
on the importance of non-governmental organizations and human rights issues including gender equality.
The aim of the 2008 High Level Forum on Aid Effectiveness in Accra was to review and reflect on the
Declaration‟s practical impacts on aid effectiveness. One consequence of the application of aid
effectiveness principles is that donors are increasingly concentrating their ODA in a limited number of
countries and sectors.
New funders and funding mechanisms – The architecture for global aid has become increasingly
complex over the last decade, with a proliferation of aid channels, the growing importance of non-DAC
and other emerging donors (such as China), and a significant degree of earmarking. According to the
International Development Association (IDA), the average number of donors per country rose from about
12 in the 1960s to about 33 between 2001 and 2005. There are currently over 230 international
organizations, funds, and programs; donor proliferation is most pronounced in the health sector (more
than 100 organizations are involved). New donors bring more resources to help developing countries
reach the MDGs, but also new challenges for harmonization and alignment.
According to IDA, about half of the ODA delivered through multilateral channels in 2005 was earmarked
by sector or theme. The boom in earmarking of aid for specific uses through global or “vertical” programs
and funds5 began in the late 1990s when several large vertical funds were created, primarily in the health
and education sectors. Vertical funds have resulted in unprecedented funding and attention for needy
causes such as infectious diseases, but critics claim they contribute to the fracturing of aid, a lack of donor
harmonization, the weakening of in-country systems, and misalignment of the priorities of donors and
recipient countries. Many vertical programs have started to take steps to better integrate with country
systems and become more “Paris-friendly”. These include the Global Fund and the GAVI Alliance which
have both signed the Paris Declaration and can be considered among the more advanced global
programmes when it comes to evaluating their practices against aid effectiveness principles, and the
Education for All – Fast Track Initiative (EFA-FTI) that expressly focuses on alignment, harmonisation
and mutual accountability.
Global conflict and security agendas – Over the past decade, an increasing proportion of global
resources have been invested in security concerns. The events following September 11, 2001, as well as
the emergence of conflicts in various parts of the world have led to massive investments in peace and
security initiatives. Currently, the conflicts in Afghanistan, Pakistan, and Iraq are absorbing considerable
global resources and the attention of world leaders, thus reducing attention paid to other developing
countries, in particular those categorized as having limited strategic importance such as Malawi.
Global economic crisis – For many developing countries, the implications of the current global financial
crisis will depend primarily on the prices of the primary commodities they export. According to the AfDB
Chief Economist, Africa‟s generally weak integration with the rest of the global economy may mean that
many African countries will not be affected by the first round of effects of the crisis. In the long run,
however, foreign investment in Africa is likely to diminish as donor countries cut their aid budgets and
foundations reduce the number and size of their grants.
5 The OECD and the World Bank define vertical programs as “international initiatives outside the UN system which
deliver significant funding at the country level in support of focused thematic objectives.” This is in contrast to the
horizontal approach of country-based aid.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
5
Evaluation Directorate – SPPB, July 2010
According to the World Bank, the short-term impact of the financial crisis on Malawi has so far been
limited, because the financial sector is small and not sophisticated. Nonetheless, in the medium to long
term, the financial crisis could have a significant negative impact through its impact on Malawi
commodity exports and international aid, as predicted by the IMF. Malawi‟s productive sector could be
severely affected through reduced demand for the country's exports, mainly tobacco, sugar, and tea; and
exchange rate movements in the west are having a negative impact on some foreign aid inflows to
Malawi.
22 .. 33 CC II DD AA CC oo nn tt ee xx tt
Over the past decade, CIDA has shifted its priorities, countries of focus, and aid delivery approaches.
Changes in countries of focus have had a fundamental effect on the scope and design of the Malawi
Country Program.
Dynamic political context and
changing priorities – During the
review period, CIDA has been
affected by various changes within
the agency and in the wider
Canadian context, including: four
elections, two Prime Ministers, five
changes in CIDA ministers, as well
as four changes in CIDA presidents.
Each has affected the agency‟s
priorities, strategies, and resource
allocations at the corporate level and
within CIDA branches. Over the
past 15 years, a number of policies
and strategies have reflected changes in priorities – the key documents are listed in the sidebar.
The overall purpose of Canada‟s
ODA has not changed since 1995
(see sidebar) and its overarching
ODA priorities (health, education,
gender, HIV/AIDS, governance, and
the environment) remained the same,
with some slight shifts in their interpretation and relative importance. This changed in 2009 when CIDA
announced new priorities: increasing food security, stimulating sustainable economic growth, and
securing the future of children and youth.
Geographic focus – Since the 2002 policy statement Strengthening Aid Effectiveness, Africa has
continued to be important for CIDA programming, but resource allocations have been significantly
influenced by Canada‟s commitments to Afghanistan (2002) and Haiti (2006), and increased interest in
Latin America and the Caribbean (2007).6 On February 23, 2009, Canada announced that CIDA “will be
focusing its efforts in 20 countries (80 percent of bilateral assistance) by concentrating resources,
focusing programming and improving coordination.” The list of countries shows an increased focus on
Latin America and the Caribbean (LAC) and a reduction in Africa. Malawi is not one of the 20 countries
6 In 2002 Canada pledged $1.9 billion for reconstruction and development in Afghanistan over ten years. Canada
committed $555 million over five years (2006-2011) to reconstruction and development efforts in Haiti. In
July 2007, Prime Minister Harper announced that “Canada is committed to playing a bigger role in the Americas.”
CIDA’s Key Policies and Strategies
Canada in the World (Canadian Foreign Policy Review), 1995
CIDA‟s Social Development Priorities: A Framework for Action, 2000
Policy Statement on Strengthening Aid Effectiveness, 2002
Canada‟s International Policy Statement: A Role of Pride and Influence in the World, 2005
CIDA Results Based Management and Accountability Framework, 2007
Sustainable Development Strategies: 1997-2000, 2001-2003, 2004–2006, 2007–2009
"The purpose of Canada's ODA is to support sustainable development in developing countries, in order to reduce poverty and to contribute to a more secure, equitable and prosperous world."
Canada in the World, 1995
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
6
Evaluation Directorate – SPPB, July 2010
of focus, but the latest indications are that CIDA will continue to provide bilateral support to the health
and education sectors.
MDGs and Aid Effectiveness – Canada endorsed the United Nations Millennium Declaration in 2000,
which led to the Millennium Development Goals. CIDA‟s Strengthening Aid Effectiveness policy
statement (2002) and Canada‟s International Policy Statement (IPS), A Role of Pride and Influence in the
World (2005) demonstrated Canada‟s commitment to the MDGs and the principles of aid effectiveness
outlined in the 2005 Paris Declaration, which resulted in international consensus around aid delivery
approaches such as results-based management (RBM) and program-based approaches (PBAs). CIDA had
adopted results-based approaches more than a decade earlier, had formalized its direction for PBAs in
2002 in Strengthening Aid Effectiveness, and published an operational guide to PBAs in 2007. While
CIDA is moving towards program and sector approaches at a policy level, most of its procedures and
management tools as well as its accountability structures are still focused on the project level. While
CIDA has made efforts to manage for results at the program level (through for example the publication of
CDPF Guidelines in 2003), the practice of managing for program level results has not been
institutionalized within the Agency to date. This is also true for sector level results; there are no
established practices in place that require CIDA programs to develop, monitor and report on sector level
strategies.
Aid Delivery Channels – Over the years, integration between CIDA‟s three delivery channels (bilateral,
multilateral, and Canadian partnership) has been very limited, mainly due to lack of guidance and also
because they are managed by different branches and with different models, mandates, partners and
priorities. Efforts have been made to increase integration (e.g., bilateral country desks have been working
closely with CPB in recent years to comment on CPB potential initiatives), but there was no joint
planning at the country level on how the three channels contribute to CIDA‟s objectives, and the lack of
integration made it difficult to define the country program boundaries. However, CIDA has recently
approved a “whole of CIDA” approach for all CPDF.
CIDA Structural Transition – An internal restructuring process, which started in 2006 and ended in
2009, has slowed down or hindered some planning and decision-making processes and has caused some
uncertainty among CIDA staff and partners with regard to branch/unit and individual responsibilities.
Malawi Country Program Context – The 10-year CIDA Malawi Country Program had three distinct
phases. The first phase (1998-2003) was characterized by the excitement of establishing a new program
and relationships. The second phase (2004 and 2005) was marked by major changes and transition in
management structure, culture, modus operandi, personnel, morale and relationships between CIDA and
its partners in Malawi which had marked negative effects on the program‟s performance. In the third
phase (since 2006), the uncertain status of Malawi within Canada‟s overall aid program, modest program
investments, and continued personnel turnover had a demoralizing effect on those working in the program
and strained relations with Canada‟s partners in Malawi.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
7
Evaluation Directorate – SPPB, July 2010
33 .. PP rr oo ff ii ll ee oo ff tt hh ee MM aa ll aa ww ii CC oo uu nn tt rr yy PP rr oo gg rr aa mm
This profile of the Malawi Country Program includes a brief history of CIDA support to Malawi, the
Malawi CPDS, and the program‟s structure and management.
Canadian Support to Malawi before 1998 – Canada‟s support to Malawi began in 1964 following its
independence. Over the next 34 years, Canada provided approximately $250 million through Africa
Branch and additional support through Canadian Partnership Branch (CPB) and Multilateral and Global
Programs Branch (MGPB). From 1964 to 1997, CIDA‟s work in Malawi was carried out through regional
projects that included Malawi. In September 1997, CIDA‟s bilateral programming shifted from a regional
focus to a country-specific focus, which led to the development of a ten-year strategy for CIDA in
Malawi.
Country Program Development Strategy, 1998-2008 – The goal of CIDA‟s 1998-2008 Country
Program Development Strategy (CPDS) was to help Malawi reduce poverty and better provide for the
basic human needs of its people by levering resources to enhance its ability to respond to new global
challenges. The Malawi CPDS was a bilateral strategy that aimed to assist Malawi in creating a healthy,
educated, and productive human
resource base as a prerequisite for
reducing poverty and attracting
investment. It outlined two sector
priorities (health and education) and
four cross-cutting priorities (gender,
governance, HIV/AIDS, and
environment). The LFA outlines
seven program level outcomes which
are described in Appendix I.
CIDA Investments in Malawi,
1998-2008 – Between 1998 and
2008, CIDA funded 452 projects in
Malawi. According to CIDA‟s
corporate data, the total ODA disbursements from all investment channels were approximately CAD
$216.6 million. These include:
Approximately $100 million (46 percent) channelled through bilateral program to 112 projects,
mostly in education and health.
Approximately $24 million (11 percent) for 52 Pan-African and sub-regional projects (mostly in
health and private sector development), which did not necessarily align their work with the
CPDS.
Approximately $64 million (30 percent) through MGPB to 146 projects, mostly in health and
emergency assistance. 7
Approximately $28 million through CPB (13 percent) to 142 projects, mostly in health,
education, democratic governance and private sector development.
7 The total disbursement figures generated from CIDA‟s corporate database need to be viewed with some caution
given that a number of the multilateral and CPB investments do not always accurately reflect the real investment
that reached Malawi.
CPDS Priorities and principles:
– Concentrate at least 80% on basic education and basic health in areas which particularly impact on girls and women;
– Maximize impact within sectors by synergistic programming, including concentration on governance issues within sectors of focus;
– Focus on increasing women's capacities and decision-making roles;
– Complement basic human needs programming with responsive mechanisms to improve governance in sectors of programming concentration and which contribute to an improved investment climate (10%) and for other emerging opportunities (10%).
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
8
Evaluation Directorate – SPPB, July 2010
The sectors receiving the greatest CIDA investment were health (45 percent of total disbursement),
education (22 percent), and governance (12 percent). Other sectors received a total of 21 percent of
disbursements. Exhibit 3.1 shows expenditures by branch and sector.
Exhibit 3.1 Disbursements by Sector and Branch (in $CAD million)
Sector of Focus
CIDA Branch
Bilateral Africa Other
8
Office of Democratic Governance
MGPB CPB Total
Democratic governance 15.7 0.5 0.06 4.2 6.1 26.8
Improving health 34.5 12.2 43.3 7.3 97 4
Private sector development 0.2 7.8 2.0 5.4 15.6
Strengthening basic education 41.4 0.1 0.18 6.3 48.2
Environment 6.3 2.78 0.37 0.49 9.9
Emergency assistance 13.6 0.06 13.6
Peace and security 0.03 0.0004 0.03
Other 1.6 0.8 0.0078 0.054 2.4 5.0
Total 99.9 24.3 0.074 63.8 28.3 216.6
CIDA Management, Funding and Oversight in Malawi – CIDA classifies its investments by delivery
channels and investment type (e.g., directive, responsive, and core funding). Recent investments in
Malawi were made through Africa Branch, CBP, and MGPB, and include all investment types. Of
particular note is the considerable variety in:
project management arrangements ranging from projects managed by GoM, by Directors
locally recruited by CIDA or by other bilateral partners;
project oversight mechanisms that included CIDA PTLs in Canada and Malawi, other CIDA
officers, other development partners (e.g., DFID), project monitors, periodic and regular audits
and evaluations; and
project funding arrangements that included co-funding arrangements (e.g., DFID), bilateral
contributions to a bilateral agency (e.g., GTZ – referred to here as a “bi-bi” arrangement), and
grants to a multilateral agency (e.g., UNICEF – referred to here as a “multi-bi” arrangement.)
CIDA also invests in non-project activities in Malawi (i.e., activities that do not have or require specific
project or program approval) that support policy dialogue, donor and sector coordination, and so forth.
These are undertaken as part of on-going responsibilities (e.g., by sector specialists and other special
advisors in the PSU, project directors and staff, and headquarters staff).
Program Management Structure – CIDA‟s program management structure in Malawi has involved four
main entities: CIDA Headquarters, the Malawi PSU, the Malawi Office of the Canadian High
Commission (since 2004), and the Canadian High Commission (CHC) which was located consecutively
in Zambia and Malawi over the review period. Responsibility for the Malawi program shifted to
8 “Africa Other” includes Pan-Africa and BFO sub-regional programming
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
9
Evaluation Directorate – SPPB, July 2010
Mozambique in September 2009. The respective roles and responsibilities of these entities have evolved
over time. Key developments and their implications for program management and performance are
discussed in section 6.9.3.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
10
Evaluation Directorate – SPPB, July 2010
44 .. EE vv aa ll uu aa tt ii oo nn AA pp pp rr oo aa cc hh aa nn dd MM ee tt hh oo dd oo ll oo gg yy
44 .. 11 EE vv aa ll uu aa tt ii oo nn QQ uu ee ss tt ii oo nn ss aa nn dd II ss ss uu ee ss
The CPE was managed by the
CIDA/SPBB Evaluation Directorate.
Data gathering and analysis were
carried out by the independent
evaluation team in close
consultation with CIDA/Evaluation
Directorate (ED). The CIDA
evaluation manager participated in
and contributed to the field visit to
Malawi in March 2009. The
evaluation team‟s overall approach
to the assignment was consultative,
participatory, and utilization-
focused. The evaluation was carried
out in conformity with the
principles, standards and practices
set out in the CIDA Evaluation
Guide (2004), the DAC Principles
for evaluations (1991), and the DAC Evaluation Quality Standards (2007) which are consistent with the
Treasury Board Evaluation and Policy Standards.
With input from CIDA, the evaluation team developed a detailed methodology and an evaluation
framework based on the Terms of Reference (see Appendix II). An abbreviated evaluation framework is
provided in Appendix III). A summary of evaluation foci is provided in the sidebar.
As requested by CIDA, the evaluation team applied a scaled rating tool to quantify performance
information on projects reviewed in detail. The Malawi CPE team made some changes to the rating
criteria in CIDA‟s generic tool to clarify how criteria would be applied at the country program and project
levels. This led to the inclusion of new criteria to: address the cross-cutting issues of governance and
HIV/AIDS, assess the realization of planned/actual outputs and outcomes as part of the effectiveness
criteria, and reduce potential overlap between the coherence and relevance criteria.
44 .. 22 EE vv aa ll uu aa tt ii oo nn DD aa tt aa GG aa tt hh ee rr ii nn gg
There were three major sources of data for this review: people, documents, and site visits. The evaluation
team reviewed numerous documents, consulted 180 stakeholders through semi-structured face-to-face and
telephone interviews, and conducted two site visits to Malawi: a one-week scoping mission during the
inception phase (November 10-14, 2008) and a two-week data collection mission during the
implementation phase (March 9-20, 2009). The list of stakeholders consulted and documents reviewed are
presented in Appendices IV and V.
The team used descriptive, content, and comparative analyses to analyze the data for this study and, to the
extent possible, ensured validity through data triangulation (confirming data from multiple sources).
Based on the data analysis, the evaluation team developed findings, conclusions and recommendations.
Program level assessment – Given the lack of program level data and the difficulty in locating program
level informants both in Canada and Malawi due to high turnover, the evaluation team made every effort
to use proxy data (i.e., data that, while not quite at the program level, were at a higher or more complex
level than individual project data). The evaluation team tapped into the corporate memory of CIDA staff
Malawi CPE Foci
Program Context
Program Relevance
Program Effectiveness
Sustainability of Results
Program Coherence
Program Efficiency and Resources Management
Principles of Aid Effectiveness
Cross-cutting Issues
Program Management, Structure and Monitoring and Evaluation
Delivery Mechanisms
Lessons Learned and Recommendations
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
11
Evaluation Directorate – SPPB, July 2010
and partner organizations that had implemented CIDA funded initiatives in Malawi, and Malawian
institutions that had benefited from a variety of CIDA funded initiatives. Despite these efforts, it proved
difficult to obtain the information required.
Investment Level Assessment –
The evaluation focused on a
sample of 18 projects (see
Appendix VI) selected during the
work planning stage. These
included CIDA bilateral,
multilateral and partnership
investments (project, program or
partner organizations). In addition,
the evaluation team reviewed
CIDA‟s support to the PSU over
the period in terms of its
contributions to CIDA program
management. The initiatives
reviewed represented 63% of
CIDA‟s investment during 1998-
2008 (approximately $135 million
out of a total of $216 million – see
sidebar).
44 .. 33 KK ee yy CC hh aa ll ll ee nn gg ee ss aa nn dd LL ii mm ii tt aa tt ii oo nn ss
Basis for measuring program, sector, and thematic performance – There is no up-to-date
performance measurement framework or tool that provides an appropriate basis for assessing the Malawi
Country Program‟s overall effectiveness. Neither the original program LFA (1999) nor the CPDS
outlined program, sector or thematic level results and related indicators that would have been suitable to
guide the evaluation. The evaluation team focussed on actual achievements and progress towards implicit
results, even if these were not necessarily outlined in the original LFA or the CPDS. In our view, using
the LFA or CPDS as the sole basis for assessment would not allow telling the full „performance story‟ of
the Malawi program.
Availability of program-level information – It was difficult to obtain information that captured CIDA‟s
program-level performance in Malawi over the 10-year period due to considerable staff turnover and
documentation that related only to individual projects. In several instances the evaluation team had to rely
on information provided in existing project documents such as progress reports and evaluations, and some
ratings are based on the team‟s interpretation of the available data. Readers should therefore consider the
project ratings as indicative rather than definitive.
Reviewing strengths and weaknesses of delivery mechanisms – The evaluation team assessed the
strengths and weaknesses of the mix of delivery mechanisms used in the Malawi Program. However,
since CIDA‟s portfolio in Malawi included only one example of a program-based approach project (its
contributions to the HIV/AIDS Pooled Fund), the evaluation team could not make a comparison with
other PBAs or make conclusive statements about the strengths and weaknesses of this and other delivery
methods.
Snapshot of Reviewed CIDA Investments
9 bilateral projects (including the PSU) representing approximately CAD$80 million in CIDA investment
9 projects implemented by 5 partner organizations, representing approximately CAD$ 55 million in CIDA investment
Of the 18 projects
– 8 projects, $ 65.5M (48%) were in the health sector (including Water and Sanitation, HIV/AIDS and Nutrition)
– 4 projects, $39.2M (29%) were in the education sector
– 2 projects, $11.4M (9%) were in the governance sector
– 1 project, $2.9M (2%) was in gender equality
– 2 projects, $3.2M (2%) were multi sectoral
– CIDA support to the PSU, representing approximately CAD$13 million (10%) of the sampled CIDA investments
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
12
Evaluation Directorate – SPPB, July 2010
55 .. SS ee cc tt oo rr OO vv ee rr vv ii ee ww
This section provides a summary of information by sector. More detailed information can be found in
Chapter 2 (Context) and Chapter 6 (Findings by Evaluation Criteria).
55 .. 11 HH ee aa ll tt hh
CIDA programming in the health sector has focused on two rather different sub-sectors: HIV/AIDS and
Water and Sanitation (WATSAN). In both areas programming has been closely aligned with sub-sector
GoM priorities and development needs.
Despite some slow improvements in recent years, health data in Malawi depict a relatively bleak
situation. Malawi has one of the world‟s highest maternal mortality ratios. HIV prevalence has been and
remains high, with severe consequences for the country‟s public health system and the overall
productivity and social fabric of the nation. In the last 10 years, HIV/AIDS has been the major health
burden for Malawi. CIDA was 3rd
of nine donors in the HIV/AIDS pooled fund (tied with NORAD and
SIDA).9
Severe drought has led to famines and food shortages in Malawi, and access to safe water remains a
serious and highly relevant challenge. In the mid-90s the Ministry of Irrigation and Water Development
(MoIWD) engaged in a renewed effort to provide access to water and sanitation, aiming at rehabilitating
the once relatively well performing Malawi water schemes, building new ones, and introducing new
approaches to ensure increased sustainability (e.g., community-based management and demand-driven
approaches). However, Malawi had very limited capacity in water management policy and governance,
and in the development and management of infrastructure – at all levels (national, district and
community). Water and Sanitation remain a priority of the GoM development strategies (e.g., PRSP
2002). In 2006, irrigation and water development were identified as one of the six priority areas in the
Malawi Growth and Development Strategy (MGDS). In 2007/08, Canada ranked 7th out of eight donors in
Malawi that supported the water sector.
55 .. 22 EE dd uu cc aa tt ii oo nn
CIDA programming in the education sector has included the supply of quality textbooks and teaching and
learning materials to the primary school system and the development of a distance education model to
train secondary teachers. Since 2007, CIDA has been investing in improving primary education through
the Initial Primary Teacher Education (IPTE) and Improving the Quality of Primary Education (IQPE)
projects.
Malawi‟s aim is to provide education for all and improve the quality of teaching. With the introduction of
free primary education in 1994 and subsequent increase in enrolment,10
providing sufficient quantities of
teaching/learning materials has been a major challenge for the Malawian government. In 1998,
procurement capacity within the Ministry of Education, Science and Technology (MoEST) and other
local bodies was basically non-existent. Increased enrolment rates at the primary level led to growing
demand for secondary education and for teachers qualified to teach at this level. As noted in the 2008
joint education sector review, secondary teacher training continues to be a priority for the GoM. In
2007/08, Canada ranked 9th among Malawi‟s 13 donors in the education sector.
9 It should be noted that the HIV/AIDS Pooled Fund in Malawi (as in most places) is dominated by the contributions
of the Global Fund, which provides 66% of the resources. CIDA as well as the GoM and SIDA/NORAD each
contributed $10 million or 5% to the Pooled Fund.
10 From 1.6 million to over 3 million enrolled students. Source:
http://www.newsfromafrica.org/newsfromafrica/articles/art_902.html
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
13
Evaluation Directorate – SPPB, July 2010
55 .. 33 GG oo vv ee rr nn aa nn cc ee
CIDA programming in the governance sector has included two major bilateral investments: the
Governance Assistance Project (GAP) and the Project for Economic Governance (PEG). Together, these
two projects focused on strengthening the roles and practices of parliamentarians, civil society, and the
GoM.
Malawi is a young democracy and its fragile political systems still need strengthening. This was
recognized in the enactment of the second republican Constitution in 1994, followed by the severance of
the Parliamentary Service from the Civil Service through the promulgation of the Parliamentary Service
Act of 1998. The ongoing need for enhanced governance is recognized by Malawi‟s political leaders who
have introduced a number of reforms and other initiatives over the past decade. Canada was one of a very
few donors to support improved governance in Malawi a decade ago, with a particular focus on the
strengthening the National Assembly, the public service, and the role played by civil society in Malawi.
Other donors, including DFID, are now active players in the sector.
In 1998, GoM processes, systems, and structures were weak (if existent). While some progress has been
made through various efforts, consulted stakeholders widely agree that the GoM remains generally weak
and understaffed. The lack of qualified staff and high turnover rates (due to competition with donor
projects/private sector jobs, and health issues – especially HIV/AIDS), pose ongoing challenges to the
work of government units, and underline the need for continued support for more effective management
within government. In 1998, civil society played a relatively passive role in Malawi, reflecting decades of
repression during the Banda era. In 2007/08, CIDA ranked 9th among Malawi‟s 11 donors in the
governance sector.
55 .. 44 GG ee nn dd ee rr EE qq uu aa ll ii tt yy
CIDA programming in the gender equality sector has focused on the GESP initiative. This fund was
designed “to strengthen the capacity of selected government and civil society organizations in Malawi to
promote gender equalities in the areas of education (promotion of girl‟s education), water and sanitation,
health (reproductive health and HIV/AIDS) and governance (promotion of women in decision-making
processes) as well as to support a reduction in gender based violence (violence against women)”.
Persistent gender inequalities pose a major challenge to women and men in Malawi and are among the
factors hampering Malawi‟s development. Inequalities are present in the political, social, and economic
spheres and at all levels from communities to Parliament and the government.
The importance of gender equality in the Malawian context is explicitly acknowledged in both the
Malawi Poverty Reduction Strategy Paper (MPRSP) and MGDS, and is emphasized as a core principle
that guides the GoM in education sector plans and strategies such as the 2001 Policy Investment
Framework (PIF) and 2008 National Education Sector Plan (NESP). Malawi developed a national gender
policy in 2000 (revised in 2008) and a national gender program in 2004. Nonetheless, technical and
organizational capacities in the area of gender are very low in both the government and civil society in
Malawi. In 2007/08, Canada ranked 5th out of Malawi‟s eight donors that supported gender equality.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
14
Evaluation Directorate – SPPB, July 2010
66 .. FF ii nn dd ii nn gg ss bb yy EE vv aa ll uu aa tt ii oo nn CC rr ii tt ee rr ii aa
66 .. 11 II nn tt rr oo dd uu cc tt ii oo nn
This chapter summarizes the evaluation findings on program relevance, effectiveness, sustainability,
coherence, efficiency, principles of aid effectiveness, cross-cutting issues, program management
(including strategic management, structure, and M&E), and delivery and funding mechanisms. These
findings draw upon analysis of bilateral projects funded by the Malawi Program over the decade, and
comparisons of their performance across criteria (see project ratings in Appendix VII).11
Overall, within the contextual challenges in Malawi, CIDA‟s program in Malawi between 1998 and 2008
was very relevant to the developmental needs of Malawi. It realized and exceeded most of the
developmental objectives as defined in the CPDS. However, the sustainability of results has varied among
projects. CIDA‟s development approach in Malawi has been generally congruent with the principles of
aid effectiveness outlined in the Paris Declaration. It has realized greatest success at the program level in
relation to the gender equality as a cross-cutting issue; its success in other cross-cutting priorities is
modest. CIDA has been more successful in striving for coherence with other development partners than it
has been within the Agency and across delivery channels. CIDA‟s potential efficiency and effectiveness
has been limited by the lack of program-level and sector strategies and monitoring mechanisms as well as
its shrinking investment portfolio since 2006.
66 .. 22 RR ee ll ee vv aa nn cc ee
Relevance refers to the extent to which the objectives of a development intervention are consistent with
beneficiaries‟ requirements, country needs, global priorities, and the policies of partners and donors
(OECD DAC). This section reviews the extent to which CIDA‟s Malawi Country Program has been and
has remained aligned with the needs and/or priorities expressed by the governments of Malawi and
Canada/CIDA.
Finding 1: CIDA’s Malawi Country Program was relevant to the development and poverty
reduction priorities of the GoM, to Malawian priorities, to the evolving priorities of the
Canadian government and sectors of CIDA focus, and was complementary to the work
of other donors.
Relevance was rated as very satisfactory in most of the projects reviewed for this evaluation (see
Appendix VII). Out of the eight criteria assessed, relevance received the second highest average rating
across all projects.
11
The main findings of the evaluation are based on a review of a sample of 18 projects for a total value of
approximately $135 million (or 63% of CIDA investments during the period). Of the 18 projects reviewed, 11 have
been reviewed in detail according to the criteria in the rating tool. The PSU projects (four phases) were reviewed
with a focus on their implications for overall program management issues (i.e., they were not reviewed against the
same criteria used for the other projects, given that PSU projects are operational rather than developmental in
nature). However, only seven bilateral projects (including the PBA in HIV/AIDS) are included in the rating grid for
a total value of approximately $69 million (or 32% of CIDA investments during the period), as two of the projects
were too new to apply the tool and because for the others the information from documents and/or stakeholders was
not detailed enough to permit a defensible quantitative judgement.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
15
Evaluation Directorate – SPPB, July 2010
Relevance to Malawi’s development priorities
Consulted stakeholders indicated that the CPDS‟ focus on helping Malawi develop a “healthy, educated
and productive human resource base” has remained highly relevant, given Malawi‟s continued severe
capacity gaps across sectors and at
all levels (see sidebar).
The CIDA Malawi Country Program
remained relevant to the priorities of
the GoM as outlined in the Malawi
Vision 2020 (2000), the Malawi
Poverty Reduction Strategy Paper
(MPRSP,2002), and to two of the six
priorities outlined in the Malawi
Growth and Development Strategy
2006-2011 (MGDS, 2006).
The country program objectives directly addressed two of the four pillars of the MPRSP (2002) – good
governance and human capital development – and CIDA‟s multilateral programming activities in
disaster/emergency response addressed a third MPRSP pillar (improving quality of life for the most
vulnerable). Both the MPRSP and the CIDA country program highlighted the relevance of HIV/AIDS and
gender as cross-cutting priorities. The country program was relevant to two of the six priorities outlined in
the MGDS (2006) – HIV/AIDS, and water and irrigation – but did not reflect the GOM‟s recent shift in
priorities to stimulate economic growth by putting more emphasis on „hard‟ issues such as agriculture and
food security, transport and infrastructure, irrigation and water development, and energy.
Relevance to Sectors
Health – CIDA programming in
the health sector was closely
aligned with GoM priorities and
with development needs in two
sub-sectors: HIV/AIDS and Water
and Sanitation. CIDA‟s
contributions to establishing the
HIV/AIDS Pooled Fund and
strengthening its capacity, and its
efforts to encourage donor
coordination through the
HIV/AIDS Pooled Fund and
program-based approaches (one of
the GoM‟s expressed preferences
for aid delivery) are seen as highly relevant and have led to progress in the sector. While the Ministry of
Irrigation and Water Development (MoIWD) has made efforts to provide access to water and sanitation,
Malawi had very limited capacity in water management policy, governance, and the development and
management of infrastructure at all levels (national, district and community). Water and sanitation have
remained a GoM priority, and CIDA‟s interventions were planned in response to these priorities and
needs. To respond to the need for increased donor coordination in the WATSAN sector, CIDA played an
instrumental role in fostering donor dialogue through the Water and Sanitation Development Partners
group and contributed to initial discussions and planning for a Water and Sanitation SWAp.
“Working with Malawi to develop an educated, healthy and productive human resource base meshes directly with objectives of access to quality health and education for all, and the development of the human resources necessary to create a manufacturing-led economy. The focus on governance, and the overall thrust of the program contributes in a long-term and more indirect way to the creation of a democratically mature nation with a fair and equitable distribution of wealth, and sustainable growth and development.” (CPDS, 1998)
In 2007/08 in Malawi, CIDA ranked
– 3rd
out of 9 donors in HIV/AIDS pooled fund (tied with a joint contribution made by NORAD and SIDA)
– 7th
out of 8 donors in the water sector
– 9th out of 13 donors investing in the education sector
– 9th out of 11 donors investing in economic governance
– 4th out of 5 donors investing in public administration
– 5th out of 8 donors investing in gender, youth development and
sports issues
Sources: Malawi Aid Atlas 2007-08, GOM 2009
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
16
Evaluation Directorate – SPPB, July 2010
Education – CIDA programming has addressed some key priorities outlined in the 2001 Malawi Policy
and Investment Framework (PIF) for the education sector and in the 2008 National Education Sector Plan
(NESP). Most noteworthy are CIDA‟s support for the development of teaching/learning materials, the
improved procurement capacity of the MoEST, and the development of effective and cost efficient
secondary teacher training, which are considered highly relevant to Malawi‟s aim to provide education for
all and improve quality of teaching. CIDA addressed existing gaps and collaborated with other donors
working in the same sub-sector to coordinate and align approaches.
Governance – Malawi is a young democracy and its fragile political systems still need strengthening.
Malawi‟s political leaders recognize the ongoing need for enhanced governance and have introduced a
number of reforms and initiatives over the past decade. While some progress has been made, consulted
stakeholders agree that the GoM remains generally weak and understaffed. CIDA‟s efforts in
strengthening the National Assembly, the public service, and the role of civil society in government
oversight (e.g., the budget review process) have responded to needs that remain important in Malawi.
Canada was one of a very few donors to support improved governance in Malawi a decade ago. Other
donors, including DFID are now active players in the sector. Finally, when CIDA‟s Malawi Country
Program began, civil society played a relatively passive role in Malawi. CIDA‟s deliberate focus on
strengthening civil society‟s role in government oversight is well recognized.
Gender Equality – As noted in the CPDS, persistent gender inequalities pose a major challenge to
women and men in Malawi and are among the factors hampering Malawi‟s development. The importance
of gender equality is explicitly acknowledged in all of Malawi‟s development plans and strategies, as well
as in its national gender policy and program. Nevertheless, capacities in the area of gender are very low in
both the government and civil society and CIDA‟s focus on gender-related capacity building, awareness-
raising, and technical assistance (through gender-focused initiatives and mainstreaming of gender in the
program‟s overall operations) has been very relevant. CIDA‟s contributions corresponded to four of the
six themes outlined in the Malawi 2000 National Gender Policy (education and training; reproductive
health, food and nutrition security; governance; and human rights). The two themes that were not
addressed are natural resources and environmental management, and poverty eradication and economic
empowerment. These thematic priorities were up-dated in Malawi National Gender program in 2005 and
subsequently in its revised Gender Policy (2008) to include HIV/AIDS and Gender Violence. CIDA
gender-equality related programming focuses on both additional themes. In keeping with Paris
Declaration principles, CIDA also played an important role in creating a common framework for
development partners to improve sector programming through the Development Assistance Group on
Gender (DAGG).
Relevance to Canada
Overall, the Malawi Country Program and individual CIDA-funded projects have been well aligned and
congruent with the development priorities of the Government of Canada (GoC) and CIDA. These include:
contributions to poverty reduction (the main purpose of Canadian ODA); CIDA‟s thematic focus on
social development, including health and nutrition, HIV/AIDS, basic education, and child protection, with
gender equality as an integral part of all of these priority areas; and Canada‟s support for strengthening
democracy and good governance. Environment was identified as a cross-cutting issue in the program
design, at a time when the emphasis was to “do no harm.” While CIDA policies evolved over the decade
to encourage more proactive efforts to address environmental concerns, the Malawi program design was
never changed nor was the program managed to reflect this new approach.
The relevance of Malawi as a development partner for Canada has varied considerably in the last ten
years. Malawi was considered highly relevant in 1997 when the country program was established and in
2005/06 when it was put on the list of CIDA priority countries. However, in 2006 priorities shifted as a
result of Canada‟s strong engagement in Afghanistan and new priorities and growing interest in the
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
17
Evaluation Directorate – SPPB, July 2010
Americas. In February 2009, Malawi was removed from the list of „countries of focus‟ and is now
considered a country „of modest presence.‟
Complementarity with other donors
CIDA‟s work in all four sectors has been complementary to that of other donors. CIDA has addressed
areas that at the time had not been supported by other donors, while also collaborating to varying degrees
with other donors working in the same sub-sectors to coordinate and align approaches, engage in donor
dialogue, and encourage others to pool resources in support of GoM objectives.
66 .. 33 EE ff ff ee cc tt ii vv ee nn ee ss ss // RR ee ss uu ll tt ss
Effectiveness refers to the extent to which a development intervention‟s objectives were achieved, or are
expected to be achieved, taking into account their relative importance (OECD DAC). This section
discusses the effectiveness of the Malawi Program.
Effectiveness in Priority Sectors
Finding 2: There is considerable evidence of positive achievements in all of CIDA’s explicit and
implicit priority sectors in Malawi. However, overall effects tend to be project- rather
than sector-oriented, reflecting CIDA program management practices in Malawi.
The Malawi Country Program had two explicit priority sectors as outlined in the CPDS (basic health and
education) and two implicit priority sectors (governance and gender equality) that were originally cast as
cross-cutting priorities. There is considerable evidence of positive achievements in these sectors at both
the project and sub-sector levels that correspond to, and in some cases exceed (e.g., HIV/AIDS), the
overall program objectives outlined in the CPDS. However, the program has not always maximized
opportunities for leveraging its investments and building on its own successes, for example in relation to
education (secondary teacher training, development of an education SWAp). Moreover, as demonstrated
below in the sector-focused findings, results tend to be project oriented, or at most sub-sector, rather than
sector-oriented. This reflects how CIDA has managed its investments in Malawi (see section 6.10).
Finding 3: In the health sector, the CIDA Malawi Program has made considerable contributions
at both the project and the sub-sector levels in the areas of HIV/AIDS and water and
sanitation. Given its focus on these sub-sectors, CIDA’s impact on the Malawi health
sector as a whole has been modest.
The Malawi Country Program did not
define program-level results in the
health sector. Its initiatives in
HIV/AIDS and Water and Sanitation
sub-sectors were not linked by an
overarching health sector strategy,
and there has been little synergy
between investments. However,
CIDA‟s programming in HIV/AIDS and Water and Sanitation contributed to tangible changes at the
community level and to creating a more conducive environment in the health sector through its efforts to
improve the capacities of the GoM and local partners, policy development, and donor coordination.
Contributions/Achievements in the HIV/AIDS sub-sector
CIDA helped to address severe capacity gaps in the Malawian National AIDS Secretariat (NAS) and the
subsequent National AIDS Commission (NAC), primarily through its involvement in establishing the
HIV/AIDS SWAp, the first program-based approach established in Malawi. It also provided
Malawi Program LFA health-related outcomes
Improved health and hygienic conditions resulting from expanded safe water supplies within an overall development strategy
Improved access to quality family planning, reproductive health and safe motherhood, and STD treatment
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
18
Evaluation Directorate – SPPB, July 2010
approximately CAD$15 million to the GoM for the implementation of the National Strategic Framework
(NSF) on HIV/AIDS.
Consulted stakeholders noted that
CIDA was instrumental in getting
the HIV/AIDS pooled fund off the
ground and bringing other donors on
board. This has contributed to the
increased capacity and improved
effectiveness of the NAC in
coordinating Malawi‟s national
response to HIV/AIDS. In addition,
many noted that CIDA‟s HIV/AIDS
Program Advisor played a crucial
role in providing technical assistance
and advice to the NAC.
CIDA supported the development of
a resource database of HIV/AIDS
activities in Malawi to identify gaps
and plan interventions more
effectively. The pooled funding
mechanism ensured a higher degree
of predictability and reliability in the
generation and transfer of
government and donor funds to the
national AIDS program, reduced the transaction costs incurred by the NAC, and provided learning
opportunities for the GoM, CIDA, and other donors on program-based approaches. Several stakeholders
expressed concern about the future of the SWAp if and when CIDA‟s support and leadership ceases.
CIDA‟s work on HIV/AIDS was complemented by other CIDA investments through CPB, MGPB, and
Africa Branch, and there is considerable evidence that these initiatives strengthened the capacities of
grassroots/community initiatives and
organizations and contributed to
regional learning and exchange.
Achievements in the Water and Sanitation sub-sector
CIDA‟s main investment in this area
was the Community Water,
Sanitation and Health Project
(COMWASH). At the institutional
level, CIDA supported the
development of the MoIWD
strategic plan (2005) through GAP
as well as the development of the
National Sanitation Policy (2008).
Further, CIDA was actively engaged
in sector-wide donor coordination
through the Secretariat of the Water
and Sanitation Development
Partners Group.
CIDA’s contributions to HIV/AIDS
CIDA supported an organizational study of the NAS that resulted in its transformation into a semi-autonomous entity (NAC) to address problems of government bureaucracy.
CIDA‟s involvement was crucial in the establishment of the Pooled Fund and related support to the evolving National AIDS Commission.
CIDA supported the development of a resource database of HIV/AIDS activities in Malawi to identify gaps and plan interventions more effectively.
Key achievements in HIV/AIDS that the pooled fund under the NAC helped to bring about
– increased number of patients receiving antiretroviral therapy
– improved care and support of people living with HIV/AIDS
– dramatic increase in the number of HIV testing sites (from 14 in 2001 to more than 400), and increased numbers of people being tested
– improved services for HIV-positive pregnant women to prevent mother-to-child transmission
CIDA’S Key achievements in water and sanitation
Increased access to safe water and sanitation in the districts of Thyolo and Phalombe:2000 water points were constructed, reaching a rural population of over 250 000; significant increase in the number of villages meeting MoIWD water access standards; consulted beneficiaries noted reduced time in water collection and fewer cases of water-related diseases
Increased capacity for community-based water management
(16,000 individuals trained in targeted communities, including almost 50% women; stakeholders expressed a strong sense of ownership and pride)
Improved GoM policies and practices in the development of the Malawi Sanitation Policy (2008), the National Water Development Program, the Training Manual for Water Users Associations, and in the emerging Water and Sanitation SWAp
Enhanced donor dialogue and coordination through the Water and Sanitation Development Partners group and initial planning for a Water and Sanitation SWAp
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
19
Evaluation Directorate – SPPB, July 2010
In targeted communities, CIDA-supported projects also contributed to positive changes related to health
issues including: reductions in under-five malnutrition and mortality rates, reduced incidence of disease
and micronutrient deficiency disorders, improved early childhood development, and increases in trained
health workers at the community level in 10 districts. (The CPE team was informed of these changes, but
was not in a position to quantify these results.)
Finding 4: In the education sector, CIDA has made significant contributions, particularly in the
areas of textbook supply and procurement and in secondary teacher education.
CIDA‟s contributions to education in
Malawi have been aligned with GoM
education sector priorities and have
filled gaps not covered by other
donors and helped to leverage
others‟ support– facts frequently
noted and valued by consulted
stakeholders. This in turn allowed
Canada to influence reforms to the
national curriculum. CIDA
supported the supply of quality textbooks and teaching and learning materials, built the procurement
capacity of MoEST (which, in turn, led to the implementation of Malawi‟s first National Public
Procurement Act), developed a distance education model to train secondary teachers, and brought
together GoM and donor representatives to discuss and prepare for establishing an education SWAp.
CIDA‟s ongoing support and advice to the GoM, the MoEST, and other education stakeholders in Malawi
is highly respected and valued.
CIDA engages in significant policy dialogue in education. CIDA is highly respected by the GoM as a
result of its expertise and capacity to work effectively and mediate solutions between stakeholders. CIDA
continues to exercise strong influence in matters related to curriculum reform, teacher training, and
textbook policies and public procurement. CIDA participates in the annual Joint Sector Review for
Education, chaired the Donor
Partners for Education Group in
2006 and 2007; belongs to five
education-specific technical
working groups; and is a member
of the Ministry‟s Program for
Curriculum Assessment and
Reform steering committee and
technical working group.
Since 2007, CIDA has been
investing in improving primary
education through the Initial
Primary Teacher Education (IPTE)
and Improving the Quality of
Primary Education (IQPE) projects.
While it is premature to assess the
contribution of these investments to
results in the education sector,
project reports provide evidence of
preliminary progress that is likely
to support the achievement of
Malawi Program LFA outcomes related to education
Growth of community schools providing equal opportunity and increased participation of communities in the education process
Improved quality of teaching and teaching/learning environment in Community Day Secondary Schools (CDSS)
Improved MoEST capabilities to supply quality gender-sensitive teaching/learning materials to schools
CIDA’s key achievements in education
Strengthened procurement capacity in MoEST: CIDA helped to establish a central procurement unit and database that allow MoEST to determine textbook requirements; strengthened the capacity of the Malawi Institute for Education to design and prepare print-ready copies of textbooks.
Improved cost-effective access to textbooks: CIDA improved the supply of textbooks and teaching materials, supported the development of gender-sensitive textbooks for primary schools, and provided special needs teaching/learning materials.
Procurement and distribution costs of textbooks are now one of the lowest in all of Africa (an average of $0.50 CAN per book). CIDA has provided 18.9 million units of teaching materials (including textbooks) through the GSES II project.
CIDA contributed to GoM policy on textbooks; textbook to pupil ratio. which was 1:24, is now 1:1
Increased number of qualified teachers working in secondary schools: Between 2003 and 2006, almost 800 teachers graduated from the CIDA-supported distance education program, which now receives government funding.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
20
Evaluation Directorate – SPPB, July 2010
expected results. These recent education projects are connected and support various components of the
National Education Strategic Plan (NESP) of the GoM, and at a broader level are linked by the Education
for All objectives and the MDGs. However, due to the lack of an overall CIDA education strategy and
sector objectives in Malawi, the projects reviewed for the CPE appear somewhat fragmented and it is not
clear whether or how they are linked to and/or build on each other.
Finding 5: In the governance sector, CIDA helped to strengthen the roles played by civil society,
the National Assembly, and individual Members of Parliament in the national budget-
making process in Malawi. CIDA support has had modest effects on targeted Malawi
government departments.
While the CPDS identified the importance of governance issues and included one outcome and indicator
that referred to governance (see
sidebar), the Malawi Program did
not define specific governance issues
that it was intended to address, nor
did it formalize a strategy, results,
and/or indicators to guide its
governance investments. Over the
decade, the Malawi Program made
two major bilateral investments
within the governance portfolio: the
Malawi Governance Assistance Project (GAP) and the Project for Economic Governance (PEG).12
Aside
from these two projects, CIDA made few other investments in governance through bilateral or other
CIDA funding channels. The program relied on assigned CIDA project officers and to personnel of
bilateral projects for governance expertise when required; the PSU did not employ any governance
specialists. CIDA did not engage in non-project activities (such as policy dialogue) in the governance
sector.
While there was some interaction between the PEG and GAP projects and some other projects (e.g.,
GESP), the Malawi Program did not have a formal strategy to guide its governance investments and there
is more evidence of project achievements than sector-level achievements. The expected sustainability of
their results (particularly GAP) is quite limited.
Together, these two projects focused
on strengthening the roles and
practices of parliamentarians, civil
society, and the GoM. The key
outputs and achievements of these
two projects are summarized below.
Increased capacity and role of civil
society in monitoring government
PEG support for civil society (CS)
included awareness raising; training
workshops; development of a pool of
trainers; facilitation of linkages
among networks, Members of
Parliament (MPs) and government
12
While some Malawi Country Program staff refer to GESP as a governance project, we have treated it in this report
as a gender project.
Governance outcome and performance indicators (1999 LFA)
Outcome: Effective policies, mechanisms and systems for sustaining governance, social and economic reforms, and addressing poverty
Performance Indicator: Productivity, living standards, economic indicators, confidence of citizens, and investors in democratic importance
Some PEG Results – Key Achievements
Enhanced collaboration between the National Assembly and CS; public participation in pre-budget consultations
CS sector networks increased knowledge and skills in budget scrutiny
GoM is more receptive to input from the public and CS
More than 120 Malawian reporters and editors trained in public finance and reporting on economic governance (Department of Journalism, University of Malawi); development of an economic journalism handbook for the Malawi media (2008)
Creation of a Civil Society/ Parliamentary task force to enhance CS engagement on Parliamentary reforms, oversight and consultation
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
21
Evaluation Directorate – SPPB, July 2010
ministries; and some institutional support.
Increased capacity of the National Assembly
PEG provided financial and technical support to Parliament, including: the cost of training for MPs,
committees and staff related to the budget process, budget scrutiny, oversight, and monitoring; support for
MP field trips related to their budget implementation oversight role; and support for evidence-based
research, equipment, and logistical support.
Some PEG Results – Key Achievements
Parliamentary Committees scrutinize poverty and social protection issues; increased quality of MP budget debates
Resources allocated to support Parliamentary Committees oversight
Parliamentary Committees meet regularly and conduct public hearings; MPs see civil society as a source of valuable budget analysis; increased public interest in such issues
The National Assembly developed a strategic plan 2006-2009; the Parliamentary Reforms Committee reviewed the National Assembly‟s performance against this plan in early 2009
PEG has supported steps to conceptualize a Parliamentary Budget Office to enhance Parliament‟s research and analytical capacity on economic policy and budgetary matters (initiative is in early stage).
Increased capacity in targeted GoM
ministries
GAP was designed in 2001 as a
responsive project to support
capacity building within the
Malawian public service. Following
a mid-term performance review that
identified several limitations, it was
redesigned in 2006 and extended to
January 2009. The project focused
on strategic planning for GoM
ministries, policy development for
the Office of the President and
Cabinet (OPC), and leadership
training for the Department of
Human Resource Management and
Development (DHRMD). GAP
contributed to a large number of
outputs but had modest success in
realizing planned outcomes,
particularly at the organizational and
institutional levels.
Finding 6: In the gender equality sector, CIDA’s contributions in Malawi are widely regarded as
remarkable, both in terms of project outputs and in CIDA’s role as a leader and
advocate for gender equality in Malawi.
The CPDS describes a number of gender objectives, including: i) to increase, through policy dialogue and
specific project initiatives, the
GoM‟s awareness of the importance
of incorporating gender analysis into
economic and policy decision-
Some selected GAP outputs
Training course on Strategic Implementation Plans (SIPs) housed within the Malawi Staff Development Institute (SDI); it is not clear if SDI has the capacity and resources to deliver and update this training on an ongoing basis.
SIPs for 21 ministries and 11 departments (including Domasi College of Education and the Ministry of Water Development); however, due to leadership turnover, many of these plans are not being used by ministries.
Deliverables related to GoM policy-making and decision-making systems: Guide to Executive Decision-Making Processes, training in policy development
Leadership development outputs: Leadership Competency Model, Leadership Facilitators‟ Guide, leadership training for 400 senior GoM managers, a Leadership Development Framework (SDI will own the framework and provide the training if it receives funding)
Gender budget guidelines, checklist, and training materials; training and orientation for government and civil society representatives (due to budgetary constraints in the GoM, the guidelines were never used for budgeting purposes).
LFA outcome: “Increased women‟s capacities and their decision making roles.”
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
22
Evaluation Directorate – SPPB, July 2010
making processes; ii) to strengthen the capacity of Malawian women as full participants and decision-
makers in development; iii) to strengthen the capacity of targeted Malawian institutions and organizations
to integrate gender-equality considerations into their policies and programs, and to promote specific
gender-equality interventions; and iv) to integrate gender-equality considerations systematically into all
bilateral projects and programs.
The Malawi Country Program‟s
contributions to gender equality as a
priority sector are the result of the
GESP project (as an investment
solely focused on gender issues),
CIDA‟s active involvement in the
Development Assistance Group on
Gender (DAGG), and the work of
the Malawi-based CIDA gender
specialists who provided technical
backstopping and policy advice to
the GoM and other partners in
Malawi. CIDA‟s efforts to
mainstream gender as a cross-cutting
issue are discussed in section 6.8.
The vast majority of consulted
stakeholders acknowledged that
CIDA‟s contributions to gender
equality in Malawi have been
remarkable both in terms of concrete
achievements as well as its role in
continuously encouraging its
partners to address gender issues in
their own work. CIDA is widely seen
as the lead donor for gender in
Malawi, not because of its relatively
modest financial contributions, but
because of its active involvement in
the DAGG, its field presence
through GESP, its full-time Gender
Equality Specialist, and its ongoing
support to the GoM and other
stakeholders. According to consulted
donors, NGOs, and GoM representatives, its work on gender has increased CIDA‟s visibility and
reputation in Malawi.
One of CIDA‟s main strengths in the gender sector is its multi-pronged and multi-level approach.
Through the work of GESP and the Malawi-based gender advisors in the PSU, CIDA has been able to
engage at different levels simultaneously (grassroots community work, lobbying, and policy
development) and with different types of partners (NGOs, MPs, GoM) on a number of thematic issues
(e.g., GBV, readmission, maternal health). This approach brought community issues to the policy level
and supported policy roll out and implementation at the community level. Consulted development
partners highly value the dedication, responsiveness, and skill of CIDA staff members (both GESP staff
and the PSU gender advisor).
CIDA’s Key achievements in gender equality
Increased capacities of government departments, district assemblies, and NGOs and CSOs working on gender issues: CIDA provided support in technical areas and in organizational capacities; NGOs noted their improved capacities to implement projects and leverage donor resources (70% of NGOs supported by GESP have obtained funding from other development partners); creation of CSO networks and strengthened lobbying capacities; creation of the National Task Force on Gender Advocacy Initiatives (NATAFOGAI)
Results at the community level: increase in the number of gender-based violence cases reported to police, increase in the number of teen mothers readmitted to school, improved access to maternal health practices, greater awareness of women‟s rights; most changes were facilitated by the involvement of local leaders, teachers, and police.
More conducive legal and political environment for gender equality: CIDA provided support for the National Gender Policy and Program, the Prevention of Domestic Violence Bill (2005), the Re-admission Policy for Teen Parents in Schools, and the Roadmap on Maternal Health.
Gender mainstreaming: CIDA supported the Ministry of Women and Child Development (MoWCD) and other ministries in the development of gender mainstreaming guidelines and strategic planning, the gender aspects of the Malawi PRSP, gender analysis of the budget and gender-responsive budgeting, and the National HIV/AIDS Policy Strategic Framework and Strategic Plan.
Improved donor coordination: CIDA supported and re-invigorated the DAGG, which has made progress in participation, communication, and clarification of roles and responsibilities of governments, donors, and NGOs; a results framework is under development. The DAGG‟s effectiveness is still limited by reliance on development partner assistance, the limited capacity of some ministries, and the limited involvement of some donors.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
23
Evaluation Directorate – SPPB, July 2010
CIDA‟s main challenges in gender programming in Malawi were due to the weak capacities of key
partners (both the GoM and CSOs), limited political will with regard to promoting reforms on gender
issues, and a parliamentary impasse that slowed the approval of gender-sensitive bills. Other limitations,
however, were related to CIDA‟s own approach. Several stakeholders were of the opinion that CIDA was
trying to do too much with relatively limited financial resources.
Finding 7: While the CIDA Malawi Program established a positive Canadian reputation among a
broad range of stakeholders in the country over the past decade, since 2006 the
uncertainty related to the future of the program has negatively affected Canada’s
reputation.
Many of the positive achievements of the Malawi Program across different sectors were due to the fact
that the program had established and maintained a positive Canadian reputation as a flexible, trusted, and
responsive donor among Malawian and international stakeholders. CIDA was also very successful in
leveraging resources of other donors to support Malawi‟s development priorities. Canada is recognized
for its proactive contributions to donor coordination and its commitment to the principles of aid
effectiveness.
Since 2006, the uncertainty of the Malawi Program‟s future has negatively affected Canada‟s reputation.
In the last three years, CIDA bilateral has made only three significant new investments in Malawi through
initiatives led by multilateral partners (UNICEF and UNDP) and bilateral partners (GTZ). The program
team in Malawi and Canada has continued to participate in events and initiatives (e.g., SWAp
developments in various sectors, development of sector-specific plans), but CIDA‟s bilateral contribution
has been limited to providing technical expertise through the Head of Cooperation and PSU sector
specialists. Interviewed development partners noted their concerns about CIDA‟s financial commitments
beyond these (valued) in-kind contributions and beyond the end of the CPDS.
66 .. 44 SS uu ss tt aa ii nn aa bb ii ll ii tt yy
Sustainability refers to the continuation of benefits from a development intervention after major
development assistance has been completed; the probability of continued long-term benefits; and/or the
resilience to risk of the net benefit over time (OECD DAC).
Finding 8: While CIDA support has contributed to an impressive number of results in Malawi
since 1998, the likelihood that these will be sustained is quite mixed and modest overall.
This reflects Malawi’s challenging context as well as the varied suitability of CIDA
practices in managing for sustainable results.
CIDA‟s programming results in Malawi are most likely to be sustained at the individual level – in the
increased knowledge and skills of planners, managers, and beneficiaries who participated in the design
and delivery of specific investments. At the organizational and institutional levels, the likely sustainability
of achievements is varied. A number of results appear to have a strong likelihood of being continued
and/or locally adapted according to evolving contexts (e.g., capacities of institutions such as the NAC,
MoEST, Domasi College; capacities of NGOs and other partners involved in GESP and PEG). In other
instances sustainability of results is doubtful (e.g., support provided to ministries and institutions as part
of GAP, and some of the support provided to the National Assembly by PEG).
Factors that affect sustainability – Malawi is heavily dependent on donors for financial resources, a
situation which may worsen given the global recession, and on external technical expertise and resources
due to the capacity constraints of the GoM and CSOs. Despite this challenging context, the evaluation
revealed some good practices that are likely to have helped support the sustainability of results. These
include:
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
24
Evaluation Directorate – SPPB, July 2010
Institutionalization: of the specialized MoEST procurement unit, the distance education model
for secondary teacher training, and gender policies developed with CIDA assistance (e.g.,
readmission of teen mothers to school, Bill and Plan of Action on Prevention of Domestic
Violence).
Organizational capacity development: Trained MoEST procurement unit staff; strengthened
capacity of the NAC to lead the national response to HIV/AIDS; strengthened community
capacities to manage their own water schemes; strengthened capacities of NGOs (in areas such as
proposal writing and RBM), networks of CSOs, and stronger connections between government
and civil society.
Ownership: Interventions based on local needs and/or built on local ideas and plans (e.g.,
secondary teacher training built on ideas developed by Domasi College; gender initiatives based
on local NGO suggestions and designs) contributed to local ownership and increase the likelihood
that Malawian stakeholders will defend and expand these achievements over time.
Diversifying Funding Sources: The HIV/AIDS SWAp assisted the GoM in diversifying sources
of funds; PEG areas of investment (civil society, media, government) attracted funding from
other agencies including UNDP, DFID, Trocaire, Oxfam, Irish Aid, EU, and others.
The evaluation also identified some approaches and limitations that hindered the sustainability of results.
These include:
Absence of explicit sustainability strategies at both program and project levels (including the
absence of comprehensive capacity development strategies) – In many of the reviewed projects,
questions of sustainability were raised too late, as the project was closing (e.g., GAP).
Catalytic but short-term investments and lack of follow-up/scaling-up strategies: Several
CIDA-supported projects (PEG, GESP, and GAP) supported a large number of interesting short-
term initiatives that are not likely to be further developed or sustained without longer-term
support.
Lack of institutional ownership of projects: In some instances, CIDA projects lacked clearly
identified local partners/counterparts. This was particularly evident in GAP in regard to strategic
plans. While the Chief Secretary of the Office of the President and Cabinet (OPC) identified
individual ministries as recipients of GAP‟s strategic planning support, these ministries were not
active partners whose institutions owned and directed the strategic planning processes (and their
results) in their ministries.
Project versus sector approach: While the Malawi government became more receptive to sector
approaches, CIDA lacked the resources to respond except by means of policy dialogue in sectors
such as education and water. Sector approaches (e.g., HIV/AIDS) could have helped build local
ownership and sustainable structures and systems.
66 .. 55 PP rr oo gg rr aa mm aa nn dd PP rr oo jj ee cc tt CC oo hh ee rr ee nn cc ee
For the purpose of this evaluation we referred to the definitions of coherence provided in the Merriam
Webster Dictionary: a) systematic or logical connection or consistency; and b) the integration of diverse
elements, relationships, or values. Both imply the existence of a describable larger „whole‟ – an
overarching common purpose or goal that individual elements can contribute or belong to.
Finding 9: While CIDA’s country program managers made some efforts to build coherence across
bilateral projects in Malawi, connections between bilateral projects have been largely
accidental. Some efforts have also been made to link bilateral priorities with CPB and
MGPB projects. This reflects the fact that the CPDS was a not a truly corporate
document that was binding on non-bilateral channels and that coherence among CIDA
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
25
Evaluation Directorate – SPPB, July 2010
program investments was not an over-riding CIDA priority during the period under
review.
Program coherence – CIDA has not defined expectations and guidelines for country program coherence,
or strategies to ensure coherence within and across sectors, or within and across delivery channels. Thus,
the limited coherence within the bilateral program reflects more on the Agency than on the Malawi
Program. The question of whether investments of different CIDA branches in a particular country can or
should be aligned has not been resolved or addressed by policy guidance. CIDA staff report that the
Agency‟s new data architecture (which should be rolled out by 2011) is expected to support greater
coherence among the branches. In the meantime, the Aid Effectiveness and Country Program Unit
(AECPU) group within CPB is working on this challenge.
Coherence among bilateral projects within and across sectors – Overall, the bilateral program was
characterized by a „silo approach‟ in which projects in different sub-sectors worked mostly in isolation. In
education, the GSES and SSTEP projects had some limited linkages in terms of their objectives/goals,
and recently approved education investments implemented by GTZ and UNICEF are linked under the
NESP. However, these are not clearly linked to an overall CIDA program or strategy for education and do
not build on previous CIDA-funded projects. In health, CIDA‟s work on the HIV/AIDS pooled fund and
the COMWASH project addressed different issues. Projects in the governance sector targeted
complementary groups of stakeholders, but did not work systematically towards explicit sector goals.
Coherence among delivery channels – The Malawi Program team and CPB made some notable efforts
to inform each other about planned and ongoing initiatives in Malawi, and MGPB had some consultations
with the Malawi Program and bilateral program specialists (Africa Branch). Synergies between bilateral
and CPB, or MGPB initiatives in Malawi were mostly accidental. This reflects the fact that there was no
policy requirement for CPB and MGPB investments to be aligned with bilateral initiatives during the
period under review. One notable exception was the complementary relationship among CPB, MGPB,
and regional CIDA initiatives on HIV/AIDS.
Project coherence – At the project level, the evaluation team examined coherence in terms of the extent
to which projects planned for complementary associations with other investments in Malawi, carried on a
regular exchange of information, and identified opportunities for synergy during implementation. There
was a significant variation across reviewed projects. The CPDS and project-level documents show the
clear intention to ensure ongoing connections among bilateral investments, but these were implemented to
different degrees and with mixed results. The longest-serving managers in the Malawi Program
encouraged regular information exchange among the staff of different bilateral projects, and while
consulted CIDA/project staff found this exchange informative, there is little evidence that this led to the
identification of opportunities for synergy among projects.
As noted previously, the Malawi Country Program lacked sector or thematic strategies. In this context, the
capacity of individual projects to identify and utilize synergies and collaborative opportunities with other
projects largely depended on the individuals involved and on contextual issues.
66 .. 66 EE ff ff ii cc ii ee nn cc yy aa nn dd RR ee ss oo uu rr cc ee MM aa nn aa gg ee mm ee nn tt
66 .. 66 .. 11 II nn tt rr oo dd uu cc tt ii oo nn
Efficiency is a measure of how economically resources/inputs (funds, expertise, time, etc.) are converted
to results (OECD DAC).
This section has two purposes. The first is to assess the efficiency of the Malawi Country Program in
terms of: a) the extent to which projects were managed efficiently; b) the extent to which the Malawi
Program had and used effective systems and procedures to manage the aid program; and c) the overall
cost of delivering the aid program in Malawi, how it has changed over time, and how it compares to other
CIDA programs.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
26
Evaluation Directorate – SPPB, July 2010
Programs require a combination of resources – human, financial, and technological – to translate goals
and objectives into results. A second purpose is to assess how Malawi Program resources have been
utilized to realize goals and objectives.
66 .. 66 .. 22 PP rr oo gg rr aa mm EE ff ff ii cc ii ee nn cc yy
Finding 10: Bilateral projects within the Malawi Country Program have a mix of efficiency ratings.
From a cross-cutting perspective, most bilateral projects were completed within budget and on time (with
a few extensions). The Malawi Country Program leveraged funding from other donors, notably in the
HIV/AIDS SWAp, but also at the project level (e.g., PEG and GESP were co-funded by DFID, and CIDA
supported the basic education project with GTZ). Donor pooling practices created efficiencies for Canada,
in sharing the relatively fixed management costs of large projects, and also created some efficiency for
Malawi by reducing the number of projects with different sets of funding criteria and processes.
Reviews of a sample of bilateral projects that received Malawi Program funding suggest a mix of ratings
– from highly satisfactory to unsatisfactory. Of all the projects reviewed, GSES II, which will end in
September 2010, has managed its budget efficiently and has exceeded expected results by using savings
from procurement activities to support unplanned initiatives. Unfortunately, evaluations of many CIDA
projects in Malawi did not assess project efficiency even though it is a CIDA requirement to do so; this
made analysis more difficult. In some instances, the evaluation team had to rely on the management costs
allocated within project budgets (which excluded in-kind support provided by CIDA for locally-managed
projects); in other cases, the evaluation team could not provide a rating because information was not
available or was outdated.
Finding 11: According to established CIDA proxy measures for program efficiency, the Malawi
Program is relatively efficient.
The efficiency of Malawi program management in the last 10 years varied over time, as shown below.
1998-2004 –. Between 1998 and 2004, the PSU was central to all CIDA activities in Malawi and,
according to those interviewed, efficiently utilized.
2004-2006 – High CIDA staff turnover reduced the time and attention that managers could give to
strategic matters, including efficient program management. Due to the lack of clearly defined roles,
procedures, and systems, program managers had to rely on ad hoc information for decision-making; and
the limited support provided by DFAIT for the Mission in Malawi meant that those responsible for
managing the program were at times consumed by others‟ responsibilities. Following an internal CIDA
review in 2005, considerable efforts were made by the program to address these shortcomings.
After 2006 – In 2007 and 2008, the Malawi operations and management (O&M13
) costs were relatively
modest (6.5% and 8.5% respectively), as were the costs of other CIDA programs that it was compared to
(see Exhibit 6.1).
The Malawi Program‟s uncertain status since 2006 has contributed to some efficiency concerns, most
notably the cost of maintaining a mid-sized PSU while the program was shrinking in size, and the
significant time Malawi program managers and staff spent on developing plans, strategies and scenarios
to inform the future of the Malawi Program. Due to lengthy decision-approval processes and the review
of the aid policy, activities such as this evaluation (initially planned for 2007) as well as other evaluations
have been delayed, and thus were received too late to properly inform CIDA programming.
13
CIDA has used this ratio for many years as a proxy measurement for assessing a program‟s efficiency.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
27
Evaluation Directorate – SPPB, July 2010
Exhibit 6.1 Comparison of O&M Costs to ODA for Selected CIDA Programs (2006/07 and 2007/08)
Year/Variable CIDA Bilateral Program
Malawi South Africa Kenya Burkina Faso
2007
ODA 13.248M 6.097M 14.160M 11.129M
O&M .340M .606M .320M .546M
O&M/ODA 6.58% 12.25% 4.42% 6.19%
2008
ODA 11.076M 8.877M 11.076M 15.534M
O&M .319M .743M .482M .627M
O&M/ODA 8.46% 10.37% 6.48% 5.13%
Finding 12: The proxy indicator used by CIDA excludes some program-related costs, which may
under-state the real costs of program management.
The O&M formula used by CIDA excludes a number of other costs that could be considered as
management costs (e.g., PSU operating costs, costs associated with DFAIT support for programs, the
costs of the CIDA HQ Program Director, Project Assistant, and CIDA HQ staff benefits and office
support costs). Including such expenses would increase the effective cost of managing CIDA programs.
66 .. 66 .. 33 RR ee ss oo uu rr cc ee MM aa nn aa gg ee mm ee nn tt
Finding 13: Despite the CPDS’ stated emphasis on sectors and themes, most of the Malawi
Program’s resources have been allocated to project needs. This has limited the
potential for strategic management, synergy, and learning within the program.
A review of human resource allocations (among CIDA staff, PSU specialists, external monitors and
evaluators engaged by the Malawi Program) suggests that resources have been allocated to respond to
project rather than sector or thematic needs and requirements. CIDA has devoted little formal attention to
defining thematic or sector strategies, monitoring progress in these areas, or to CIDA learning across a
sector or theme, which has limited the potential for synergy.
For example:
Projects that might have shared cross-cutting objectives related to governance (e.g., PEG, GESP,
GAP) were not necessarily managed or overseen by the same CIDA officer.
PSU-contracted sector specialists with expertise in gender and water were available to support
CIDA (or project) needs if/as requested to do so. However, they did not have any authority to
provide guidance to projects, and did not have oversight responsibilities even if they were
responsible for a cross-cutting issue such as gender. (Note: No PSU advisory support was
available for governance or HIV/AIDS.)
In most cases, external monitors were allocated by project rather than thematic priorities. The
external monitors for GESP (gender) and for two governance projects did not have any
responsibilities to examine the rest of CIDA‟s portfolio in Malawi in relation to gender or
governance, even though these were crosscutting themes.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
28
Evaluation Directorate – SPPB, July 2010
Finding 14: Until 2004, CIDA managers fostered cooperation and coordination among all
stakeholders involved in the Malawi Program to build the program. Since 2006, a
significant amount of staff time has been used to manage existing investments and
generate requested information to inform the program’s future.
Between 1998 and 2004 – CIDA managers fostered close cooperation and coordination among CIDA
bilateral projects, among CIDA and other donors, and between CIDA and the GoM. The common shared
vision and values and close relationships of this period are evident in project documents and in CIDA‟s
leadership role and active participation in coordination committees.
Between 2004 and 2005 – Following changes at the Post and PSU in Malawi leadership took a more
rigid management approach and discouraged sharing of knowledge and information among project teams.
This contributed directly to high staff turnover and created tensions between CIDA and some of its
partners in Malawi. This situation abated by 2005.
From 2006-2008 – In a context of limited resources, CIDA representatives in Malawi continued to
manage previously approved projects. They focused on rebuilding previously strained relationships and
continued to coordinate with donors, the GoM, and with Canadian partners in Malawi. They also invested
significant resources to address the needs of management for information that might determine the future
of the Malawi Program.
Finding 15: Canada’s initial investment in establishing a PSU greatly facilitated the start up of its
aid program in Malawi in 1998. The relative and potential added value of this
investment has declined since 2006.
The creation of the PSU in 1998 was an important ingredient in the successful launch of the Malawi
program and PSU specialists helped to enhance program performance and Canada‟s reputation in
education, water, and gender equality up to 2004. In 2005, given the expected growth of the program, the
new PSU Director was given some additional responsibility to assist in new program development.
However, with the new program context following 2006, CIDA no longer needed this type of support
from the PSU and the PSU infrastructure started to exceed the program‟s needs. The future of the
program was unknown and dependent on policy decisions beyond the program managers‟ control, and,
while the PSU experts continued to provide their expertise and analysis, their involvement in new
programming was limited given the context. This situation contributed to certain inefficiencies.
Finding 16: After 2004, DFAIT provided some administrative support services to CIDA in Malawi,
but the lack of a formal and well understood agreement caused tensions and over time
contributed to some inefficiency in the program.
Prior to 2004, CIDA relied on the PSU for administrative support in Malawi. In 2004, a decision was
made to establish a CIDA mission in Lilongwe that would be staffed with one Canadian based staff
(CBS) and two Locally Engaged Staff (LES). In a series of email correspondence which started in 2003,
CIDA and DFAIT began to discuss an
arrangement whereby DFAIT would
provide “minimal support” to the
mission in Lilongwe with the help of
the Canadian High Commission (CHC)
in Lusaka, Zambia. According to those
interviewed in CIDA and DFAIT, the
minimal support arrangement was
intended to provide CIDA with access
to a very narrow array of common
services. It was quite distinct from the
DFAIT Support for Operations and Missions Abroad
DFAIT has the mandate to manage the procurement of goods, services and real property at missions abroad. These common services are mandatory for departments to use when required to support Canada‟s diplomatic and consular missions abroad. These services are optional for departments to use when required for purposes other than in support of diplomatic and consular missions abroad (Inter-departmental Memorandum of Understanding on Operations and Support at Missions Abroad p. 1 April 2004).
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
29
Evaluation Directorate – SPPB, July 2010
larger package of services typically provided to larger missions; however, the details of this package were
never formalized in terms of an agreement.
In 2005, the CIDA representative in Malawi and the DFAIT representative in Lusaka started to negotiate
an agreement. The arrangement was neither completed nor signed, in part due to larger decisions related
to Malawi (i.e., the transfer of responsibility for Malawi from Zambia to Mozambique, and the closing of
the mission in Malawi in 2009). A final observation relates to the expectations of the CBS based in
Malawi at the time, who indicate that they were not aware of the previous informal agreement between
their departments about the “smaller package” of services. This information gap created some tension
among individuals in both departments, who evidently did not share the same understanding of the
“arrangement”.
Between 2004 and 2008, the administrative requirements of the mission in Malawi increased significantly
for several reasons:
Given the need for enhanced security in missions abroad, it was decided that the mission in
Lilongwe required upgrading. While DFAIT was responsible for managing the construction
project, it had no representative based in Lilongwe for oversight purposes. As a consequence,
those physically present in Lilongwe (particularly the CBS) took on some responsibilities with
adverse effects on their workloads. Final activities related to this project were completed in the
summer of 2009 following the decision to close the mission.
In 2005, CIDA decided that it required a second CBS in Lilongwe to support its programming in
Malawi, although the actual posting was deferred until 2007. This had implications for the
amount of office space, housing, and other support required by mission personnel.
Over the period there were some consular cases and diplomatic visitors that required some short-
term support by Canada‟s representatives in Malawi.
During this period of considerable change, CIDA program staff in Malawi became increasingly involved
in administrative activities that they felt needed to be addressed and that in other larger missions would
normally have been the responsibility of DFAIT. CIDA officers reported they felt that they were spending
excessive amounts of their time on DFAIT responsibilities,14
while DFAIT officers were under the
impression that they were only to provide minimal support. The lack of a formal agreement contributed to
tensions between CIDA and DFAIT. During the period, there was considerable turnover in staff
responsible for the Malawi program at CIDA in Gatineau and in Lilongwe. The combination of the
informal arrangement with DFAIT, the absence of a signed agreement, and the turnover in staff likely
contributed to a loss in corporate memory of the original implicit agreement between CIDA and DFAIT.
A review of CIDA files indicates that significant and frequent efforts were made by CIDA and DFAIT to
identify, discuss and address CIDA‟s evolving needs. One notable effort to improve the situation was the
development of An Action Plan: To improve administration at CHC Malawi. This Action Plan was
created in 2007 and identified key issues to be resolved as well as the individuals responsible for the
various tasks. Considerable time was spent trying to clarify responsibilities, correct mistakes and resolve
oversight issues, which led to some improvements in areas such as financial management. However, one
central element was the establishment of a Service Standards Agreement between Lilongwe and Lusaka
that was never finalized.
In February 2009, DFAIT issued a new Inter-departmental Memorandum of Understanding on Operations
and Support at Missions. Persons interviewed for this evaluation indicate that many of the changes made
in the 2009 agreement were influenced by the lessons learned in Malawi (as well as other countries where
DFAIT has provided “special” common services to missions abroad). One important clause in the 2009
14
For example, between 2005 and 2007, the sole CBS reported that she spent from 60-80% of her time on DFAIT
responsibilities.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
30
Evaluation Directorate – SPPB, July 2010
agreement is the need for any unique operating arrangement or partner-specific program issues to be
described in a partner-specific annex. Such a measure should go a long way in preventing a recurrence of
the challenges noted above.
66 .. 77 PP rr ii nn cc ii pp ll ee ss oo ff AA ii dd EE ff ff ee cc tt ii vv ee nn ee ss ss
Finding 17: CIDA’s development approach in Malawi has been generally congruent with the
principles of aid effectiveness outlined in the Paris Declaration at the project level but
less so at the program level.
While the design of CIDA‟s program in Malawi predated the principles of aid effectiveness outlined in
the Paris Declaration (2005), there is evidence that the program has been managed in keeping with those
principles at the project level, and to a lesser extent at the program level.
The majority of reviewed projects had highly satisfactory ratings in terms of ownership, alignment and
harmonization. At the program level, there were some positive achievements, such as the CPDS design
process, CIDA‟s active role in donor coordination and harmonization, and so forth.
CIDA‟s generally positive record has been negatively affected over the past three years due to the lack of
timely official communication with the GoM and donors, who had not anticipated CIDA‟s program
decision. According to the Joint Donor Evaluation on Managing Aid Exit and Transformation “Good
partners keep each other well informed of their respective intentions and plans.”
There was some dissonance between CIDA management approaches in Malawi and several of the
principles of the Paris Declaration (particularly ownership, harmonization, and mutual accountability).
Ownership: During the CPDS design stage, CIDA conducted broad consultations with stakeholders in
Malawi and sought GoM ownership during the design of individual projects. CIDA supported projects
that focused on capacity development of GoM departments and entities (e.g., GSES II, COMWASH,
HIV/AIDS support), supported SWAps and other pooled fund arrangements that could be led by the
GoM, and encouraged the active involvement of Malawi government ministries in project oversight (e.g.,
the MoWCD). Several GoM stakeholders noted that CIDA‟s plans were not always congruent with its
practices (e.g., the use of external consultants and/or Canadian executing agencies rather than GoM
resources). However, this reflects the practices common at the time of program design, which preceded
these principles.
Alignment: The evidence suggests that the CPDS was aligned to the priorities identified by the GoM in
its main development strategies. The CIDA Malawi Program and individual projects were designed on the
basis of reviews of existing national strategies, priorities, and needs, using extensive consultations with
GoM and other local and international stakeholders. Some projects were directly related to specific
Malawi strategies for water and sanitation (COMWASH) and gender (GESP).
Harmonization: CIDA played an active role in donor coordination and harmonization in Malawi (e.g.,
HIV/AIDS SWAp and preparation for other possible SWAps) and in developing sector coordination
groups. Other donors have recognized CIDA‟s neutrality and consider it predictable and trustworthy.
CIDA‟s positive record in donor coordination and harmonization has been weakened in recent years by its
limited financial commitments for SWAps. Interviewed stakeholders noted a disconnect between CIDA‟s
ability to generate good ideas and provide the resources required to put them into operation.
Results: All CIDA partners in Malawi acknowledge the Agency‟s focus on results, and credited it for
having supported RBM capacity building among its partners (e.g., through GESP, within the DAGG).
However, CIDA has been more effective at utilizing these approaches at the project level, rather than
sector, thematic or program levels. Additional details are found in Section 6.9.4.
Mutual Accountability: At the project level, CIDA shared project reports and evaluations with the
relevant ministries and other key stakeholders. At the program level, CIDA reported regularly to the GoM
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
31
Evaluation Directorate – SPPB, July 2010
on financial expenditures but not on developmental results at the program, sector, or thematic levels on a
formal, regular basis. CIDA‟s limited investments in Malawi after 2006 combined with its limited
consultations and communications to determine the future of the program in Malawi raised concerns
among its partners about its future assistance plans. Lessons from other donors highlight the importance
of open dialogue and a respectful process that recognizes mutual accountabilities. Following the recent
decision on geographic focus, the Malawi Program has been having discussions with the GoM regarding
programming plans.
66 .. 88 CC rr oo ss ss -- cc uu tt tt ii nn gg II ss ss uu ee ss
This section examines how effectively the Malawi program has integrated CIDA‟s cross-cutting themes
of gender equality and environment, as well the cross-cutting themes of HIV/AIDS and governance (that
were established as priorities in the CPDS) into program management practices in Malawi over the
decade.
Finding 18: At the project level, CIDA management paid most attention to gender equality; other
cross-cutting issues received mixed levels of attention.
In individual projects and initiatives, CIDA made efforts to address the four cross-cutting issues identified
in the CPDS: environment,
governance, gender, and HIV/AIDS.
The extent to which these were
addressed at the project level has
varied
Gender – Consulted stakeholders
commended CIDA for its attention
and commitment to gender
mainstreaming, and there is
evidence that CIDA-supported
projects made significant efforts in
this area.
In most projects reviewed, gender
was addressed in planning (in
defining results, and some projects defined gender-related indicators, strategies/activities, and/or allocated
resources to address gender concerns) and in project monitoring and reporting. Key challenges in gender
mainstreaming, as reported by consulted staff and stakeholders, include: i) mixed levels of partner
capacities, awareness, and commitment to gender equality; some ambiguities regarding the role and
authority of CIDA‟s gender expert vis-à-vis individual projects; and the limited financial and other
support required to operationalize the gender strategy.
HIV/AIDS – There is evidence that HIV/AIDS was mainstreamed in some projects, particularly in the
education and gender sectors (e.g., GESP project selection criteria included integration of HIV/AIDS;
GSES II paid specific attention to how HIV/AIDS was addressed in the Life Skills materials; SSTEP
supported the creation of a gender and HIV committee at Domasi College and modules were introduced
on gender equality and HIV/AIDS in education curricula). However, these efforts were isolated, and did
not take place within an overarching framework for HIV/AIDS mainstreaming and there was no program-
level tracking or reporting of progress or results.
Environment – When the Malawi CPDS was developed, CIDA ensured that all projects would be subject
to an environmental assessment. Since then, more priority has been given to promoting sustainable
environmental management and behaviours. The CIDA PSU water sector specialist in Malawi was
responsible for assessing all new projects according to the standards of the Canadian Environmental
Gender Mainstreaming in Malawi Projects
SSTEP – planned for 35% of participants to be women and facilitated their participation; established a gender committee at Domasi College and supported gender sensitization classes
HIV/AIDS Program Support – integrated a gender approach in the project MOU and Contribution Agreement; CIDA‟s PSU Gender Specialist assisted in integrating gender into annual work plans
COMWASH – Aimed for 50% women in scheme committees and training; monitored the number of women participants; trained facilitators to engage villages in identifying gender issues
PEG – Formulated a gender strategy to guide project activities; work plans and reports paid attention to gender issues
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
32
Evaluation Directorate – SPPB, July 2010
Assessment Act (CEAA). This mainly concerned COMWASH and Canada Fund projects; other CIDA
initiatives in Malawi were “soft” initiatives that were not seen to have a considerable physical impact on
the environment. To our knowledge none of the reviewed CIDA projects in Malawi has gone beyond
compliance with the CEAA principles (e.g., by initiating proactive measures related to environmental
management).
Governance – The CPDS planned to complement basic needs programming (at least 80% of resources)
with 10% of resources to be spent on responsive mechanisms to improve governance in programming
sectors. In all priority sectors CIDA has worked with the respective GoM ministries, usually supporting
policy development and/or assisting with capacity building (e.g., COMWASH worked with the Ministry
of Water, GESP with the Ministry of Women, SSTEP and GSES with the MoEST, and the HIV/AIDS
with NAC). However, none of the reviewed projects (other than those explicitly focused on governance
issues) developed governance-related indicators, results, or reporting.
Finding 19: The CIDA Malawi Program did not establish specific objectives or strategies or
allocate resources and responsibilities to systematically address cross-cutting issues at
the program level. This makes it difficult to track and assess the program’s overall
effectiveness in mainstreaming these issues.
With the notable exception of gender, the CPDS made no specific mention of program-level strategies
and CIDA did not establish frameworks, guidelines, or tools to support the mainstreaming of cross-
cutting issues at the sector and/or program level. Resources for addressing cross-cutting issues at the
program level were quite limited and ad hoc.
Gender – The CPDS for Malawi had a strong gender focus and an articulated gender strategy, but this
was not accompanied by defined results/expectations or indicators, and the program did not develop a
framework to operationalize the strategy. An additional challenge was the absence of clear leadership and
accountability for gender mainstreaming (as for other cross-cutting issues). The Malawi program‟s focus
on health and education was based on the (implicit) belief that these sectors had direct implications for
women and girls. The program‟s strong gender focus reflected the beliefs and values of the program
manager at the time, a gender expert participated in program design and initial roll out, and all CIDA and
PSU staff in Malawi received annual training in gender and HIV. CIDA employed a gender-sector
specialist in the PSU who provided advice and technical information on gender to CIDA project teams
and the DAGG only.
HIV/AIDS – The Malawi Program treated HIV/AIDS as a programming sector and never defined an
explicit strategy to address this as a cross-cutting issue across investments. A PSU specialist was
responsible for both gender and HIV from 2000 to 2004, but no one assumed responsibility for
HIV/AIDS after her departure. CIDA staff frequently called upon the Malawi-based Program Advisor for
the HIV/AIDS project to provide advice on HIV/AIDS-related issues, but his role did not include
responsibility for mainstreaming HIV/AIDS in the Malawi Country Program.
Governance – The CIDA Malawi Program never defined a program-wide strategy for mainstreaming
governance across individual investments, nor did it allocate dedicated resources (such as a governance-
sector specialist) to mainstream governance in its programming. There are no established tools or
practices to track or report on governance results across the program, and accountability for
mainstreaming governance in the program is not defined.
Environment – In the CPDS, environment was mentioned as a cross-cutting issue in a short sidebar.
Former CIDA staff members were surprised to hear that environment was a cross-cutting program issue;
given the program‟s focus on social development, environment was not an obvious concern.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
33
Evaluation Directorate – SPPB, July 2010
66 .. 99 PP rr oo gg rr aa mm MM aa nn aa gg ee mm ee nn tt ,, SS tt rr uu cc tt uu rr ee ,, aa nn dd MM oo nn ii tt oo rr ii nn gg aa nn dd
EE vv aa ll uu aa tt ii oo nn
66 .. 99 .. 11 OO vv ee rr vv ii ee ww
This section examines how the CIDA Malawi program was managed. It considers the strategy used to
manage the program, how the program was structured, how results were tracked and how risks were
managed.
66 .. 99 .. 22 SS tt rr aa tt ee gg ii cc MM aa nn aa gg ee mm ee nn tt
Strategy typically considers how the overall vision, objectives, and program direction align with the
internal and external context and resource allocations. In the case of Malawi, the 1998 CPDS outlined the
strategic program directions for the period 1998-2008.
Finding 20: To launch a country program in Malawi in 1997-1998, CIDA crafted a focused,
distinct, innovative, and bold development strategy that was carefully designed to
maximize the impact of Canada’s relatively modest resources.
In September 1997, following a decision to shift from regional to country-specific support to Malawi,
CIDA initiated a process to develop
a country program development
strategy (CPDS) to guide its bilateral
programming investments in
Malawi. It identified two clear
priorities (health and education)
which were congruent with the
Malawi Vision 2020, four cross-
cutting priorities (gender,
governance, HIV/AIDS, and
environment) and promised a 10-
year commitment to Malawi. For a
donor that was re-entering Malawi,
this strategy was rather bold,
innovative, and risk-taking.
The CPDS also defined a list of project selection criteria that CIDA would use to maximize the use of its
limited financial resources. These included identifying opportunities for leverage and synergy with other
projects and donors, and close donor coordination/cooperation. It also highlighted the need to go beyond
projects and support policy dialogue in Malawi. These strategies resulted in positive impacts for several
Malawian organizations that benefited from leveraged Canadian resources, and for Canada‟s reputation in
Malawi, particularly among donors.
Finding 21: The 1998 CPDS guided CIDA’s bilateral investment decisions in Malawi up to 2004.
Since 2004, it has had less influence over the program’s investment decisions and
directions, particularly since there were limited resources for new programming.
Like most CIDA country program strategies, the Malawi strategy focused only on bilateral investments.
Between 1998 and 2004, the CPDS clearly guided CIDA investment decision-making in Malawi. The
CPE team found that the majority of CIDA‟s bilateral investments over the period were congruent with
CPDS priorities and principles. Those managing the program paid considerable attention to the cross
cutting priorities and project selection criteria identified in the CPDS.
CPDS Priorities and Principles
Concentrate at least 80% of investments in basic education and basic health
Maximize impact within sectors by synergistic programming, including concentration on governance issues within sectors of focus
Focus on increasing women‟s capacities and decision-making roles
Complement basic human needs programming with responsive mechanisms to improve governance in sectors of programming concentration and that contribute to an improved investment climate (10%) and for other emerging opportunities (10%)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
34
Evaluation Directorate – SPPB, July 2010
Deliberate efforts were made to ensure that cross-cutting priorities (in particular gender) were
well integrated in each project design (e.g., gender in COMWASH and PEG) and/or efforts were
made to encourage project staff involved in one project to interact with those involved in other
projects (e.g., GAP, PEG, and GESP as well as GESP and COMWASH).
Canada played an active role in donor and GoM coordination in several areas including gender,
education, HIV/AIDS, and water, and involved sector specialists when appropriate. CIDA also
played a leading role in governance; the creation of the media group and the Parliamentary Task
Force are examples. PEG was the Secretariat for the Parliamentary Task Force until 2008.
Canada was successful in generating support for its initiatives from other donors.
Thus, there is considerable evidence that the CPDS informed CIDA‟s bilateral decision-making in
Malawi until 2004. However, since 2004, the CPDS has had limited influence on CIDA‟s investments
and actions in Malawi. This is due to a variety of factors inside and outside the CIDA Malawi Program.
In 2004, a number of changes occurred in the Malawi Program context including turnover of
several individuals who had played key roles in both defining and managing the program until
then. In addition, CIDA experienced considerable challenges with the introduction and staffing of
old and new program positions in Canada and Malawi after 2004. During this period, CIDA staff
focused their energies on managing existing investments and responding to internal challenges
and crises. After 2004, the bilateral program made relatively few new investments in Malawi.
Thus, there was limited need to use the CPDS.
A mid-term review of the Malawi Country Program in 2004 15
flagged the need for some
refinements to the CPDS but these were never addressed due to the program context identified
above, which created some questions about the continued relevance of the CPDS. Between 2006
and 2008, the uncertain status of the program further diminished the relevance of the CPDS.
There was little clarity on priorities (explicit or implicit), which limited the scope of program
managers to make decisions including about new program investments.
In conclusion, the CPDS has not been used as a guiding instrument by CIDA for the Malawi Program
since 2004, nor has it been used to measure the program‟s performance
Finding 22: The Malawi Country Program LFA has had limited utility as a basis for capturing and
measuring the program’s expected performance beyond the project level.
The Malawi program LFA (1999) was intended to provide the basis for guiding its investments and
reporting its progress over the next decade but was of limited utility for measuring program performance:
While the program had clearly defined sector priorities (health and education) and cross-cutting
priorities, the expected program outcomes in the LFA are defined more broadly and are difficult
to measure (e.g., increased women‟s capacities and their decision-making role).
In some cases, there is a one-to-one alignment between program outcomes and the outcomes of
individual project investments. While this helped to ensure that project investments were tied to
overall program outcomes, it was less effective in capturing intended sector-level outcomes that
went beyond any one project.
The LFA sometimes includes multiple outcomes for one sector (e.g., three of the seven identified
outcomes were related to education).
15
“Mid-Term Country Program Review of the CIDA Malawi Program, 1999-2009”. Michael Miner for CIDA
(November 2004)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
35
Evaluation Directorate – SPPB, July 2010
The LFA outcomes did not provide the best basis for guiding or measuring the success of the program at
the sector or thematic levels or overall. The 2004 mid-term evaluation noted various limitations with the
LFA, and recommended a number of revisions. This led to a revised draft LFA that was never approved
or used, largely as a consequence of program upheaval after 2004.
Sector and thematic levels – The Malawi Program lacked sector- and thematic-level monitoring and
reporting systems and information that could be used to assess its effectiveness at such levels. However,
this information was never required by CIDA, so the limitation lies with the system itself, rather than with
those managing the Malawi Program.
While the Malawi Program defined several sector and cross-cutting thematic priorities in the CPDS, the
distinction among sectors and themes varied considerably over time. Similarly, the frameworks,
strategies, and processes it used to monitor progress at these levels were fragmented and less
systematically defined. For example:
The program lacked defined strategies to guide and monitor sector and thematic priorities,
although the CPDS did include a section entitled “gender strategy.”
Initially, the program formally identified education and heath as its two key sectors. While these
foci were never formally modified, education, governance, HIV/AIDS, water, and gender equality
evolved as the “de facto” foci for program staff. Moreover, the sector „home‟ for projects was not
always commonly shared or understood (e.g., some viewed GESP as a governance project while
others saw it as a gender project).
The program engaged technical specialists in the PSU for some priority areas (gender, education,
and water16
) but not others (governance, HIV/AIDS). The implicit and explicit roles and
responsibilities of technical specialists in the PSU varied considerably in terms of monitoring at
sector or thematic levels.
External monitors reported on projects rather than sectors.
Country program level – There are
no established results-tracking
systems in place in CIDA that
require country program managers
to track and report on the
developmental and operational
performance of programs on a
regular periodic basis.17
While
CIDA program managers prepare
annual reports that are consolidated
into CIDA‟s Annual Report to
Parliament, the report format is
generic and Agency-wide and is not
intended to report on a country
program‟s planned/actual performance. Interviewed country managers and directors indicate that they
kept track of the Malawi Program‟s performance through informal means including periodic discussions
with staff. While such measures are essential for day-to-day program management, they are less helpful in
tracking and documenting program performance and lessons learned.
16
The water specialist also advised the program on environmental matters as required.
17 This is distinct from other corporate reviews such as CPE and program audits that are commissioned by others.
2004 Mid-Term Review
Objectives: to review results achievement and sustainability of results in the context of the CPDS and the program LFA, the relevance of the program and its effectiveness in leveraging resources, an analysis of the complementarity of CIDA assistance with that of other donors, and the quality of CIDA partnerships with program stakeholders.
Conclusions: The review concluded that a combination of astute management and a well-integrated program focused in four sectors resulted in a well-positioned and well-regarded CIDA program in Malawi. It recommended that the program should be maintained in its present form and identified nine areas for improvement.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
36
Evaluation Directorate – SPPB, July 2010
At the program level, CIDA commissioned two external program evaluations of the CPDS over the past
decade: the Mid-Term Evaluation in 2003/04 and this evaluation in 2008/09. An analysis of the
recommendations of the 2004 mid-term review (see Appendix VIII) indicates that seven of the nine
recommendations were partially, mostly, or fully addressed. The recommendation to create a new position
in the Malawi program was not addressed, largely due to the uncertain status of the program. The
recommendation that the program establish and maintain an appropriate program LFA so that the
consultant who conducts the review at the end of Year nine of this ten-year CPDS will be able to establish
whether the Malawi Country Program has achieved all its planned outputs and several of its outcome76
was marginally addressed: the LFA was updated but was never formally approved.
66 .. 99 .. 33 PP rr oo gg rr aa mm MM aa nn aa gg ee mm ee nn tt SS tt rr uu cc tt uu rr ee aa nn dd LL ee aa dd ee rr ss hh ii pp
Programs, like organizations, require structures to transform investments (human, financial, or other) into
plans and actions. Management structures for programs are typically designed with three objectives in
mind: to clarify key roles and responsibilities; to clarify reporting, oversight, authority, and accountability
relationships; and to clarify communication and coordination responsibilities for the key units or actors
responsible for managing the program. Effective program management also requires leadership and
guidance – be it through individuals or teams/units.
Over the past decade, the program management structure for the CIDA Malawi Country Program spanned
five countries (Malawi, Canada, Zambia, Zimbabwe, and most recently, Mozambique) and multiple
management units, and its leadership has changed hands numerous times. Prior to 1998, CIDA‟s support
to Malawi was managed out of CIDA Headquarters with field representatives in Zambia (1964-1991) and
in Zimbabwe (1991-1998).
Finding 23: The CIDA Malawi Program’s somewhat atypical management structure allowed the
program to become operational quickly. However, the approach was not sustainable
and created some lasting management difficulties and risks.
Program management 1998-2004 – A program manager at the CIDA Malawi Desk managed the
program, supported by the PSU and the PSU Director (a CIDA officer with significant diplomatic
experience who acted as the de facto Head of Cooperation). Staff employed by the PSU had dual
reporting relationships to the country program manager in Canada and to the local PSU director.
The country program team was small and easy to manage and, individuals interviewed for this study
report that the streamlined management structure worked and had several positive effects on the CIDA
Malawi Program: it became operational very quickly, and within two years, six large bilateral projects
were up and running; it was flexible and responded quickly to new opportunities.
However, the program‟s management structure in this period was not typical of CIDA practice at the
time, particularly in regard to roles, reporting, and accountability. In practice, PSU advisors assumed
different roles over time due to their skills/abilities and experience. During this period, some established
CIDA practices related to contracting and delegated authorities were not followed.
Program management 2004-2006 – In 2004, Canada established an Office of the Canadian High
Commission in Malawi, a Head of Cooperation (HoC) was appointed, and CIDA initiated several
changes to the Malawi Country Program structure and engaged new individuals in Malawi. The culture
that had made the atypical structure work was gone, and the implications, difficulties, and risks
resulting from the ambiguities in the roles and responsibilities, accountabilities, and reporting
relationships among the Desk, Post, PSU, and others became quite apparent. In 2005, CIDA
commissioned an internal review to identify needed changes and most of the recommendations were
subsequently addressed. During this period, there was also a high turnover of several individuals in
leadership positions in Canada and Malawi.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
37
Evaluation Directorate – SPPB, July 2010
Program management 2006-present – The Canadian High Commissioner to Malawi (resident in
Zambia) departed in June 2007 and the HoC in Malawi was invited to serve as Acting High
Commissioner in August 2007. The office of the CHC in Malawi had two Canadian and two local staff
and the Malawi Program was supported by the PSU. This situation had lasting effects on the management
of the CIDA Malawi Program. Those interviewed for the evaluation say that there are continued
ambiguities about the role and authority of the PSU, the role of PSU sector specialists, and the role of
project team leaders (PTLs) in the implementation of projects managed locally in Malawi (which poses
some potential risks due to the possible perception of conflict of interest).
On 31 August 2009, the office of the CHC in Malawi was permanently closed and the GoC decided to
relocate responsibility for the Malawi Program to Mozambique. A modest in-country PSU will remain in
Malawi while a CIDA officer stationed in Mozambique will conduct monitoring visits to Malawi. This
“hub and spoke” approach will maintain required accountability for Canada‟s investments in Malawi.
Finding 24: A vacuum in CIDA leadership in 2004/05 negatively affected the overall performance
of CIDA’s Malawi Program. This situation may be an anomaly, or it may be indicative
of problems faced by other small and less-visible CIDA country programs.
Due to the management challenges described above, the high turnover, and the long delays in filling some
positions, there was a leadership vacuum in the Malawi program and attention was focused on the
practical needs of getting things done as opposed to more strategic considerations. The situation required
some senior-level analysis and diagnosis as well as strong leadership and decisive actions – none of which
was forthcoming. This had immediate and long-term negative effects on program performance including
its effectiveness and efficiency.
This situation in Malawi may be an anomaly, or it may be indicative of problems faced by other small and
less-visible CIDA country programs. CIDA should be vigilant to avoid similar situations in the future.
66 .. 99 .. 44 MM oo nn ii tt oo rr ii nn gg ,, EE vv aa ll uu aa tt ii oo nn ,, aa nn dd RR ii ss kk MM aa nn aa gg ee mm ee nn tt
Finding 25: CIDA’s RBM and risk management principles have been widely respected at the
project level, but less so at the program and sector levels.
Project level – The Malawi Program invested considerable time and money in project-level reporting,
monitoring, and evaluation over the past decade. All projects have dedicated significant attention to RBM
and risk management – they have applied RBM principles and tools, and have put in place appropriate
project management systems and resources and risk identification and mitigation processes..
All reviewed projects earned highly satisfactory ratings of their RBM and M&E practices. All projects
were required to submit regular financial and narrative reports; most bilateral projects had assigned
external project monitors; the majority of bilateral projects were subjected to mid-term or final
evaluations; and Annual Project Performance Reports (APPRs) or their equivalents were prepared for
bilateral projects. Consulted stakeholders indicate that monitoring reports, operational reviews, and
evaluations have been used to address implementation challenges and steer selected projects in new
directions (e.g., GAP) or to define new project phases to address important gaps (e.g. building sustainable
capacities in the Ministry of Education for textbook procurement). An unintended but positive benefit was
that the use of external monitors (who typically served long terms) provided continuity and corporate
memory during times of staff turnover. Moreover at least one project (GESP) also developed their
partners‟ capacities in RBM.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
38
Evaluation Directorate – SPPB, July 2010
Risk management – CIDA had
various mechanisms in place,
including the use of external
monitors, to identify risks at the
project level. At the program level,
however, its risk management had
several deficiencies in planning,
reporting, and accountability. The
program‟s LFA did not identify
assumptions and potential risks in relation to the Canadian government, CIDA, or the Malawi program –
all of which posed risks and ultimately had implications for the program. Most of the reviewed Program
Performance Reports for Malawi did not include risks and assumptions (see sidebar for one notable
exception) and it was not clear who was managing risks within the program, particularly in 2004/05. Due
to turnover and gaps in leadership and accountability, checks and balances were not in place or were not
used.
66 .. 11 00 PP rr oo gg rr aa mm II mm pp ll ee mm ee nn tt aa tt ii oo nn aa nn dd DD ee ll ii vv ee rr yy MM ee cc hh aa nn ii ss mm ss
CIDA implemented and delivered the Malawi Program over the past decade using a variety of investment
types, management mechanisms, delivery channels, and oversight mechanisms. While important
experiences have been gained, the CPE review sample was too small to permit the evaluation team to
make general conclusive statements about the strengths and weaknesses of the various delivery methods.
CIDA has been involved in only one functional PBA in Malawi to date, and the evaluation sample
included only one project implemented by a Canadian Facilitation Agency. The analysis of delivery
models provided in Appendix IX summarizes information from consultations with stakeholders,
document review, and our perceptions of the particular strengths and weaknesses of the delivery models.
Finding 26: The Malawi government’s limited experience with program-based approaches,
combined with CIDA‘s limited financial resources for new programming after 2006
warranted CIDA’s use of a mix of program implementation and delivery mechanisms
in Malawi.
The investment in the HIV/AIDS SWAp was the lone PBA in Malawi. Although donors and developing
countries are paying increased attention to program-based approaches, which are in keeping with the
principles of the Paris Declaration, the evaluation found that the Malawi program‟s mix of approaches
was very appropriate in the Malawi context during the period.
PBAs – Given Malawi‟s considerable human resource and organizational capacity constraints in 1998, it
had limited capacity to enter into PBA-like arrangements. CIDA was instrumental in supporting one of
the first successful PBA mechanisms in Malawi, the HIV/AIDs pooled fund. (In 2008, a mid-term review
of the only other SWAp in Malawi, the Health SWAp, raised some concerns about the GOM‟s ability to
honour its financial commitments and about the Ministry of Health‟s ability to negotiate with donors. 18
)
Although CIDA had limited resources to entertain new investments in SWAps after 2004, it helped lay
the groundwork for two new SWAps in education and water in Malawi.
Mix of approaches – Most consulted stakeholders expressed the continued need for a mix of different
delivery and funding mechanisms to provide complementary types of assistance and to spread investment
risks in Malawi. When PBAs are not possible, donors can still have positive influence in a sector through
individual projects, and a mix of short-term and long-term funding mechanisms can help donors respond
to the absorptive capacities of partners.
18
Malawi Health SWAp Mid Term Review Summary Report, NORAD, January 2008
Risk Analysis – Education Sector
In the education sector, a joint process was undertaken in which all major donors involved in primary and basic education participated. Canada‟s responsibility was for the production of the Malawi Education Sector Integrated Risk Assessment, May 2007. This identified the risks, prerequisites, recommendations and the next steps in a variety of areas.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
39
Evaluation Directorate – SPPB, July 2010
77 .. CC oo nn cc ll uu ss ii oo nn ss ,, LL ee ss ss oo nn ss LL ee aa rr nn ee dd ,, CC oo rr pp oo rr aa tt ee
CC oo nn ss ii dd ee rr aa tt ii oo nn ss aa nn dd RR ee cc oo mm mm ee nn dd aa tt ii oo nn
77 .. 11 CC oo nn cc ll uu ss ii oo nn ss
The findings of this evaluation need to be understood within the context of the Malawi Program‟s
evolution over the past decade. Between 1998 and 2004, the program went through a “golden period”
characterized by the excitement associated with the establishment, growth, and development of a new
program. The period was marked by a focused Country Program Development Strategy, high program
capacity to absorb and access financial resources for programming, strong Canadian leadership, close
collaboration with development partners in Malawi, staff continuity, as well as a very positive Canadian
profile and reputation.
The next phase (2004 and 2005) was one of transition and transformation. It encompassed changes in
the structure of Canada‟s presence in Malawi (with the opening of a Canadian High Commission), high
turnover of several individuals in leadership positions in Canada and Malawi, increased formalization of
program policies and procedures, limited growth in programming, as well as staff morale concerns and
turnover.
The final phase (2006-09) was distinguished by uncertainty regarding the future of Canada‟s program
in Malawi. Faced with a shrinking bilateral program portfolio, the program staff managed existing
investments and devoted significant time to generate information to inform the program‟s future.
Canada‟s positive reputation and high profile diminished with the GoM and among its development peers
in Malawi. At the time of writing, the Malawi Program is going through a major transition and the latest
indications are that CIDA will continue to provide bilateral support to the health and education sectors.
CIDA’s Program Performance in Malawi
The Malawi Country Program has performed well in terms of results and the relevance of its financial
investments and technical expertise in health, education, governance, and gender equality. It was very
successful in leveraging resources of other donors to support Malawi‟s development priorities. It gained
considerable respect from the Malawi government and other donors and partners for its leadership in
gender equality and HIV/AIDS and for its role in encouraging and supporting donor coordination.
Over the period, CIDA was a consistent and strong advocate for increased donor harmonization,
effectiveness, and accountability and was respected by the GoM for its expertise and capacity to work
effectively and mediate solutions between stakeholders. CIDA was an early proponent of PBAs in
Malawi, and has provided funding and technical advice in the development of PBAs in HIV/AIDS,
education, and water and sanitation. It has led thematic coordination groups on water, HIV/AIDS, gender
equality, and education, and has engaged in significant policy dialogue.
The main shortcomings of CIDA‟s Malawi Program relate to the lack of program-level and sector
strategies to plan, manage, monitor and report on its performance and manage risks and the sustainability
of some project results.
Future Directions
As a program in a country of modest presence, the future Malawi program will likely have a smaller
budget and a smaller infrastructure in Canada and abroad. It may want to consider whether some
established practices for countries of focus (such as the development of a detailed CPDS) are still
necessary. It will be important for managers to identify efficient ways to manage the transition from a
core program to one of modest presence.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
40
Evaluation Directorate – SPPB, July 2010
77 .. 22 LL ee ss ss oo nn ss LL ee aa rr nn ee dd
The key lessons learned are summarized below.
Concerning country program design and management:
To be effectively managed, programs require deliberate strategies, accountabilities, mechanisms,
and resources for planning, monitoring, and reporting at both program and sector levels.
Concerning managing for sustainable results:
The sustainability of project results is more likely when sustainability is addressed at the project
design stage and considered throughout implementation, not in the last year(s) of a project.
Project managers are more likely to pay continued attention to sustainability of results if this is
identified and managed as a priority in project and program management practices.
Concerning program transition:
Early warning and clear communication with key stakeholders are necessary ingredients in
respecting good partnership principles and fostering a smooth transition when there is a major
change in program status.
Concerning CPE design and management:
A CPE can assess only those dimensions and concepts in a meaningful way that have actually
been relevant for and have been used by (or could have been used by) the respective program
(e.g., the concepts of program coherence and efficiency).
Carrying out several CPEs at the same time can create opportunities for synergy, as it is likely
that all evaluations will be struggling with similar issues. Opportunities for synergy can be lost
however if: i) scheduling does not allow time for systematic exchange between evaluation teams,
and ii) commonalities between different CPEs are merely assumed in prescriptive methodological
guidelines, but are not backed up by empirical evidence or actual experiences gained during the
evaluation process.
A CPE is most valuable when conducted at a point in time where the findings and
recommendations can actually be used to inform planning and decision-making regarding the
program‟s future.
77 .. 33 CC oo rr pp oo rr aa tt ee CC oo nn ss ii dd ee rr aa tt ii oo nn ss
The overall good achievements of the Malawi Program notwithstanding, the evaluation identified areas
where improvements can be made to enhance this performance. Some of these pertain to broader Agency-
level improvements, such as the issue of whole-of-Agency approach, the strengthening of performance
management processes and the guidance on managing country program transitions. Considering that the
Agency has taken decisions on these aspects, and actions are underway to address them, they will not be
the subject of specific recommendations. Actions being taken by the Agency includes:
As indicated in CIDA‟s Management Response to the Office of the Auditor General of Canada
(OAG), the Agency has agreed on the need for better whole-of-Agency alignment and coherence
through the different channels and established a new CDPF process through which that can be
addressed. Regarding implementation and monitoring, discussions are underway to determine the
most effective and efficient approach.
Also, as indicated in CIDA‟s Management Response to the Office of the OAG, the Agency has
taken steps to strengthen its performance management process: The Country Development
Programming Framework (CDPF) requirements for countries of modest presence would be
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
41
Evaluation Directorate – SPPB, July 2010
lighter than for countries of focus, and the CDPF guidelines have been modified to reflect this.
The CDPFs also lay the ground for better managing a program for results. Expected results,
indicators and targets related to priority sectors mapped to CIDA‟s thematic priorities are
identified in CDPFs and are monitored and reported on as part of the program performance
management process.
CIDA‟s Aid Effectiveness Action Plan consolidates the Government priorities of focus,
efficiency and accountability with our international commitments on aid effectiveness under the
Paris Declaration (2005) and the Accra Agenda for Action (2008). Each program develops
specific targets and actions that are reflected in C/RDPFs and Performance Measurement
Frameworks (PMFs). Discussions are underway to provide further guidance on managing country
program transitions.
77 .. 44 RR ee cc oo mm mm ee nn dd aa tt ii oo nn
Given uncertainties about the status of CIDA‟s future program in Malawi at the time of the evaluation, the
TORs for the Malawi CPE excluded an analysis of CIDA future programming directions in the country.
For this reason, the Evaluation Team does not provide any recommendations to the Malawi Program
related to its future strategic direction.
One recommendation is addressed to the program on management issues. – The Malawi Program
remains in transition from a core program to one of modest presence. It is generally expected that the
program budget will be much smaller than before, with a smaller program infrastructure in Canada and
abroad.
Recommendation 1 : The managers of the CIDA Malawi Program should identify efficient ways to
deliver this modest presence country program.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
42
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx II LL FF AA (( 11 99 99 99 )) Narrative Summary Expected Results Assumptions/Risk indicators
Program Goal:
To help Malawi to reduce poverty and to better provide for the basic human needs of its people by levering resources to enhance its ability to respond to new global challenges
Impact:
(1) Reduced poverty and more balanced distribution of wealth in areas of GOM-CIDA cooperative initiatives
(2) More simplified and transparent political, business and investment environment
(3) Significantly more women as decision-makers in, and beneficiaries of political and economic society
Improved health care, particularly for women and girls
(4) Equitable access and equality of opportunity for boys and girls to quality education
(5) Stronger capacity to balance environmental protection and economic growth
Performance Indicators:
(1) Statistical indicators of poverty, wealth and income distribution
(2) Stability and openness of political and investment environment
(3) Number of women in positions of influence and actively participating in society and government
(4) Health, population statistics, demographic data, hygienic conditions, quality, availability of safe water
(5) Education statistics, tracer studies, economic indicators (disaggregated)
(6) Economic and environmental indicators, statistics
Political environment remains open to further strengthening of key democratic institutions
Economic performance, growth remain positive; - GOM remains committed to poverty alleviation
GOM and CIDA priorities and commitment to economic and social development continue to be complimentary
Risks:
Democratic foundation is fragile and business environment is not predictable
Barriers to women's participation and progress are not easily overcome
HIV/AIDS will seriously impact on economic productivity and GOM social programs
Program Purpose:
To assist Malawi to create a healthy, educated and productive human resource base
Outcomes:
(1) Effective policies, mechanisms and systems for sustaining governance, social and economic reforms and addressing poverty
(2) Increased women's capacities and their decision-making roles
(3) Increased access to quality family planning, reproductive health and safe motherhood and to STD treatment
Performance Indicators:
(1) Productivity, living standards, economic indicators, confidence of citizens, investors in democratic process
(2) Change in number, levels of women in decision-making positions and degree of active participation in national and community activities
(3) Health statistics, mortality, conception, fertility rates, STD statistics
GOM remains open to gender equality and implementation of equality initiatives
Developmental efforts of international agencies will be carried out in partnership with GOM
CIDA's initiatives with GOM will be integrated into GOM structure and procedures
Working with communities and districts will promote, contribute to sustainability of results
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
43
Evaluation Directorate – SPPB, July 2010
Narrative Summary Expected Results Assumptions/Risk indicators
Program Purpose:
(cont‟d)
(4) Improved health and hygienic conditions resulting from expanded safe water supplies within an overall development strategy
(5) Growth of community schools providing equal opportunity and increased participation of communities in the education process
(6) Improved quality of teaching and teaching/learning environment in CDSSs
(7) Improved MOE capabilities to supply quality gender sensitive teaching/learning materials to schools
(4) Health statistics for areas where safe water supplies were installed; dev't plan progress
(5) Quality, quantity, appropriate location of community schools, equality, breadth and gender balance of community participation
(6, 7) Feedback from students, educators, results on exams, monitoring of schools, assessment of quality
CIDA and Malawi CP priority sectors continue to be health, education, water, good governance
Risks:
Management, technical capacity and resources are limited
Prevalence of HIV/AIDS will seriously affect the ability of Malawi organizations, personnel to fully participate and sustain results achieved
Resources:
Management and administrative services
PSU services
Malawi CP IPF : $48.5 million for the period 99/00 to 2002/2003
Contribution of GOM and its Ministries and agencies
Malawi's inputs, staff involvement as partners
Outputs:
Projects/initiatives to meet targets of:
80% of resources dedicated to creation of a healthy, educated and productive HR base
10% of resources to improved investment climate - 10% of resources to emerging opportunities
Measurable progress and results achieved from projects and initiatives according to annual work plans
Performance Indicators:
Progress towards obtaining percentage objectives carried out semi-annually
Quality and quantity of results obtained in each area - Amount of resources invested
Number of initiatives completed and measurement of results obtained
Number and % of new initiatives planned and/or undertaken
Relationship of initiatives to Malawi's and CIDA's priorities
Relationship of training completed to that planned - Quantity, quality of training results
Applicability of training to roles and responsibilities
GOM has capacity to sustain the results of GOM-CIDA initiatives
Coordination agreements and mechanisms are in place to support Country Program Delivery Strategy
Planning and implementation have to consider the effect of limited resources and absorptive capacity of GOM
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
44
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx II II AA bb bb rr ee vv ii aa tt ee dd TT ee rr mm ss oo ff
RR ee ff ee rr ee nn cc ee
CIDA Malawi Country Program Evaluation – Abbreviated Terms of Reference
Introduction
In October 2008, CIDA contracted Universalia Management Group to conduct an external evaluation of
its Malawi country program. The evaluation will pursue three major objectives:
1) A summative review of the Malawi Program investments during the 1998-2008 period
2) An assessment of the performance of different investment types (i.e. different „delivery
mechanisms‟) including various projects, as well as the Canadian contribution to the
Government of Malawi Program Based Approach (PBA) in HIV/AIDS.
3) An analysis of overarching findings and lessons.
The evaluation will cover the ten-
year implementation period of the
Country Program Development
Strategy CPDS from 1998/99 to
2007/08 (see also sidebar).
The proposed approach to the
evaluation has been developed
during an evaluation design phase
(September – December 2008) that
included a Scoping Mission to
Malawi (November 10-14, 2008). During the design phase, a broad number of CIDA staff and
development partners in Malawi were consulted to inform the evaluation methodology.
Methodology
Overall approach: The evaluation will be carried out in conformity with the principles, standards and
practices set out in the CIDA Evaluation Guide (2004), the DAC guidelines for evaluations (1991), the
DAC Evaluation Quality Standards (2007) and, to the extent possible, the Government of Malawi‟s
specified „code of conduct‟. Stakeholder participation will be an integral component of planning;
information collection; the development of findings; evaluation reporting; and results dissemination.
Investment Sample: Reviewing the performance of individual CIDA investments in Malawi will be an
important source of information for the country program evaluation. Given that the evaluation timeframe
and resources would not allow an in depth review of all of CIDA‟s investments, the evaluation will focus
on a carefully selected sample. 18 projects covering a spectrum of different sectors, delivery and
management mechanisms were selected based on a set of criteria. These projects cover bilateral
investments, as well as those of CIDA‟s Canadian Partnership and Multilateral and Global Programs
Branch.
CIDA’s 1998-2008 Country Program Development Strategy
(CPDS) outlines the agency‟s overarching objectives for its support to Malawi over the period 1998-2008. According to the CPDS, the goal of CIDA assistance to Malawi was to help Malawi to reduce poverty and to better provide for the basic human needs of its people by levering resources to enhance its ability to respond to new global challenges. Over the last 10 years, CIDA's bilateral program aimed to concentrate on assisting Malawi to develop its human resource base as one of the prerequisites for reducing poverty and attracting investment.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
45
Evaluation Directorate – SPPB, July 2010
Evaluation Framework: Based on the review of the evaluation TORs, project background documents, and
consultations with CIDA and program stakeholders in Malawi and Canada, we have developed a draft
evaluation framework. According to the evaluation purpose and objectives, it conceptualizes the
assignment into the following dimensions:
i) Context, ii) Relevance, iii) Effectiveness/Results, iv) Sustainability, v) Coherence, vi) Efficiency, vii)
Management Principles, viii) Cross cutting issues (gender equality, environment, HIV/AIDS and
governance), ix) Monitoring and Evaluation, x) Strengths and Weaknesses of different Delivery
Mechanisms, and xi) Lessons learned and Recommendations.
Universalia has developed a detailed evaluation matrix that elaborates key evaluation questions to be
addressed under each of these areas at program and project levels, as well as indicators and data sources.
Data sources: The evaluation will use three main sources of data: People, Documents, and Observations
during the evaluation field mission (March 9-20, 2009). Key data collection methods involving people
(e.g. representatives of the Government of Malawi, of other donor organizations working in Malawi, of
NGOs, as well as staff in CIDA HQ and Post in Malawi) will be: Individual interviews, small group
interviews, Focus groups, and E-mail consultations.
Data Collection Tools: Semi structured interview protocols will be used to guide data collection through
interviews. These protocols can be shared with the respective stakeholders in advance. Findings deriving
from document review will be captured using project summary sheets structured along the same questions
as outlined in the evaluation matrix.
Data Analysis will incorporate descriptive, content, and quantitative analysis approaches. In all these
methods, reliability will be ensured through data triangulation (using convergence of multiple data
sources), the use of standardized instruments and compliance with standard practices in evaluation.
Reporting: The evaluation team will share first impressions and observations with CIDA Malawi at the
end of the field mission in an informal debrief. Within eight weeks of returning from the field mission, the
evaluation team will submit a draft evaluation report for review by CIDA. Within two (2) weeks of
receiving CIDA‟s comments on the draft report, the consultants will submit a final evaluation report
including an abstract/executive summary. CIDA will share the evaluation findings with program
stakeholders.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
46
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx II II II AA bb bb rr ee vv ii aa tt ee dd PP rr oo gg rr aa mm
EE vv aa ll uu aa tt ii oo nn MM aa tt rr ii xx aa nn dd PP rr oo jj ee cc tt RR aa tt ii nn gg TT oo oo ll
Program Level Matrix
The following table provides a summary of the key evaluation questions used to assess the performance
of the Malawi Program. It is based on the approved detailed Evaluation Matrix found in Appendix II of
the Technical Report.
Evaluation Dimension
Program Level Evaluation Questions
0. Context What have been key developments/changes in the program‟s external context (global, in Canada, in Africa, in Malawi)?
What have been key developments/changes in the program‟s internal context?
How have these changes affected program performance?
1. Relevance Has the program been and remained relevant in relation to
The Government of Malawi (GoM) Growth and Development Strategy (MGDS) and its predecessors?
Malawi national sector priorities and strategies?
CIDA‟s objectives and strategy in Malawi (CPDS)?
CIDA‟s corporate poverty reduction and sustainable development objectives?
The Millennium Development Goals?
2. Effectiveness Results Achievement
Does the overall Country Program have sufficiently clear objectives?
Overall Progress
What progress was made towards the achievement of expected results (at output, outcome, impact levels) as outlined in the program LFA and the CPDS?
What were the main factors influencing the achievement or non achievement of results?
Is there evidence of the program having contributed to poverty reduction in Malawi?
What changes in key Malawian partner institutions has the program contributed to?
What unintended results (positive or negative) has the Malawi program generated?
3. Sustainability To what extent are program results sustainable and/or present a good potential for sustainability?
Did/do the local institutions have the human and financial capacity to sustain and continue the progress/the results achieved in the key sectors addressed by the program?
4. Coherence How effective has CIDA been in ensuring program coherence?
What have been the key factors affecting program coherence?
What processes were put in place at CIDA HQ and in Malawi to ensure the overall coherence of the program? How did the program manage
The internal coherence of the program (between individual projects/ between sectors)?
The complementarity of delivery channels (multilateral, bilateral, partnership)?
The complementarity with other donors?
Program coherence with Canadian objectives abroad.
To what extent were potentials for synergies identified and used effectively? What, if any, opportunities for synergy were lost?
To what extent have different investments aiming to support/strengthen the same Malawian institution (e.g. the NAC) been coherent?
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
47
Evaluation Directorate – SPPB, July 2010
Evaluation Dimension
Program Level Evaluation Questions
5. Efficiency At the program level, have existing human, technical and financial resources been used efficiently?
How do the administrative costs of the Malawi program compare with those of other similar CIDA programs?
To what extent has the program taken measures to enhance overall program efficiency (e.g. in planning, managing, monitoring, evaluating program investments)
How does CIDA efficiency compare with other „benchmark‟ donors in Malawi?
6. Management Overall use of management principles at the program level
To what extent were CIDA management principles of local ownership, harmonization, alignment, result-based management and risks management applied at the program level?
To what extent were program activities undertaken within the perspective of the Paris Declaration?
To what extent were program management systems and resources appropriate, timely and effective in view of managing the program for results?
What have been key challenges in view of program management? What have been strengths/supportive factors?
How appropriate was the structure/design of the Malawi country program? Have the resources put into investment design been used effectively?
7. Cross Cutting Issues
How effectively has the Malawi program integrated CIDA‟s cross-cutting gender and environment themes?
To what extent were gender and environment themes taken into account during/in view of
preliminary analysis?
policy dialogue?
the implementation process ?
monitoring and evaluation of results achievement?
To what extent have program level objectives in cross cutting areas as described in the CPDS been achieved?
8. Result management / Monitoring and Evaluation
What systems and procedures for monitoring, evaluation, and reporting at the program level were put into place? To what extent were they used? Were they appropriate and sufficient?
To what extent did CIDA engage in joint monitoring activities with other donors? How effective and appropriate were these?
To what extent where RBM principles applied effectively and in meaningful ways at the program level?
To what extent were recommendations of the program mid-term review relevant, appropriate and to what extent have they been addressed?
To what extent were the findings of project level monitoring and/or evaluations/reviews used to inform program management?
What processes were put into place to identify and manage program risks?
9. Strengths and Weaknesses of different Delivery Mechanisms
What types of delivery mechanisms (funding and management mechanisms) has CIDA used in Malawi? (Including PBA)
To what extent were delivery mechanisms selected based on a clear and appropriate rationale?
What have been strengths and weaknesses, opportunities and threats related to the different delivery mechanisms utilized?
To what extent were adequate resources (human, financial, technical, other) in place to adequately select, set-up, manage, monitor and report on each delivery mechanism?
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
48
Evaluation Directorate – SPPB, July 2010
Evaluation Dimension
Program Level Evaluation Questions
10. Lessons Learned and Future
Lessons Learned
What lessons can be learned with regards to:
Program design, management and reporting
The coherence of different CIDA investments at the country level
The strengths and weaknesses of different delivery mechanisms/ complementarity between delivery mechanisms
The key strengths and weaknesses of CIDA‟s program/overall engagement in Malawi
Leadership, influence and leverage of the Canadian cooperation in Malawi.
Future/Recommendations
What changes to present strategies, priorities and practices are recommended?
What are the main challenges that should be considered in future programming?
How can the strengths and weaknesses of the current programming be used to improve the program‟s overall performance?
How to better assess the impact of Canadian-assisted projects and programs in Malawi in the future?
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
49
Evaluation Directorate – SPPB, July 2010
Project Rating Tool
Generic Project Rating Tool: For the CPE exercise, CIDA developed a generic rating tool to help
standardize the processes used to rate CIDA projects across CPE teams.
Revisions to the Project Rating Tool: During the preparation of CPE workplans, some CPE teams
(including the Malawi team) proposed changes to the generic rating tool; these were aimed at clarifying or
finessing the rating criteria proposed in the generic tool. Proposed changes were reviewed and approved
by CIDA before they were used by the CPE team during the evaluation process. As a consequence, there
is some variation in the criteria used across CPE teams. For the Malawi CPE, the main substantive
changes are as follows:
The number of criteria for most dimensions was increased from 3 to 4, to reflect missing elements
(e.g. relevance to the work of other donors was added as an additional criterion to the relevance
dimension).
The effectiveness criteria were revised to focus on the realization of planned outputs and
outcomes (rather than the effectiveness of a project in relation to poverty reduction,
sector/thematic objectives and gender equality).
Cost-effectiveness was excluded as a criterion for analyzing effectiveness as the required data to
carry out such an assessment was not available.
The sustainability criteria were revised to include evidence of measures taken by project
proponents to address sustainability of results and evidence of stakeholder ownership of key
results.
The coherence criteria were revised to reduce potential overlap with the relevance criteria and to
accentuate demonstrated actions taken to encourage synergies and complementarities among
relevant projects within the portfolio.
Criteria were developed for each of the four cross-cutting issues to increase the specificity of the
guidance in the generic tool and to complement what was provided in that tool (e.g. to clarify
criteria used for governance and HIV/AIDS).
Criteria for the efficiency dimension were revised to increase the specificity of the guidance in
the generic tool.
Rating criterion: Ratings for most dimensions (relevance, effectiveness, sustainability and cross-cutting
issues are based on four criteria. The exceptions are coherence, efficiency and CIDA management
principles and RBM/Monitoring and Evaluation, which are based on three criteria.
Rating scale: Ratings for dimensions are based on either a four or five point scale ranging from 0 (very
unsatisfactory) to 4 (highly satisfactory) as indicated on the following page. In cases where the
assessment dimension is not applicable, the review team gave the rating NA= not applicable. When the
reviewers did not have sufficient information to make a judgment, the review team indicated not
rated=NR.
Application: The rating tool was applied to the reviewed bilateral investments included in the evaluation
sample (including the PSU projects). Available information on CPB and multilateral investments was not
detailed enough to permit assessment.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
50
Evaluation Directorate – SPPB, July 2010
Rating Criteria
Dimension Assessment Criteria Rating Rating Criteria
1. Relevance Relevance in relation to Government of Malawi‟s (GoM) sectoral needs and priorities as expressed in documents such as the Growth and Development Strategy (MGDS) and its predecessors, as well as in respective national sector priorities and strategies (if they exist).
Relevance in relation to CIDA‟s objectives and country strategy in Malawi (CPDS)
Relevance in relation to CIDA‟s corporate poverty reduction and sustainable development objectives
Relevance in relation to the work of other donors (e.g. project sought complementarities, avoided duplication)
4 Highly satisfactory If all four criteria apply
3 Satisfactory If three criteria apply (must include relevance in view of GoM‟s needs and priorities)
2 Moderately Satisfactory If two criteria apply (must include relevance in view of GoM‟s needs and priorities)
1 Unsatisfactory If one criterion applies
0 Very unsatisfactory If none applies
2. Effectiveness/
Results
Progress towards envisaged outputs
Progress towards envisaged outcomes.
Exceeding expected results at output level
Exceeding expected results at outcome level.
4. Highly satisfactory The project has achieved or has exceeded most/all of its outputs and most or all of its outcomes.
3 Satisfactory The project has achieved or has exceeded most/all of its outputs. There is considerable evidence of progress towards some outcomes.
2 Moderately Satisfactory The project has achieved most/all of its outputs. There is moderate evidence of progress towards some outcomes.
1 Unsatisfactory The project has achieved some results at the output level. There is no evidence of progress towards outcome level results.
0 Very unsatisfactory There is no evidence of progress towards the projects envisaged results at output or outcome levels
3.Sustainability Evidence of the project having deliberately worked towards ensuring sustainability of results. (E.g. existence of sustainability strategy, deliberate choice of strategies to foster sustainability, regular identification and
4. Highly satisfactory If all four criteria apply
3 Satisfactory If three criteria apply
2 Moderately Satisfactory If two criteria apply
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
51
Evaluation Directorate – SPPB, July 2010
Dimension Assessment Criteria Rating Rating Criteria
mitigation of risks to sustainability).
Malawian units/organizations/institutions express/demonstrate ownership of key results.
Individuals or teams in Malawian partner organizations/institutions have the required knowledge and skills to sustain and dynamically adapt results.
Malawian partner organizations/institutions have resources and adequate processes/practices/structures in place that support the potential sustainability of results.
1 Unsatisfactory If one criterion applies
0 Very unsatisfactory If none applies
4. Coherence Project design and/or workplanning documents make explicit reference to complementarities with other relevant CIDA investments in Malawi.
Evidence of relevant information exchange occurring on regular basis between this and other relevant CIDA investments in Malawi.
Evidence of project identifying and using opportunities for synergies with other relevant CIDA funded projects during its implementation.
4. Highly satisfactory If all three criteria apply
3 Satisfactory If two criteria apply
2 Moderately Satisfactory If one criterion applies
1 Unsatisfactory If none applies
5. Cross-cutting issues
5.1 Gender Gender equality issues were satisfactorily addressed during preliminary analysis conducted in preparation of the project.
Gender equality issues were satisfactorily addressed when engaging in policy dialogue with Malawian partners.
Gender equality issues were satisfactorily addressed during the project implementation process.
Gender equality issues were satisfactorily addressed in the monitoring and evaluation of project results achievement.
4. Highly satisfactory Gender equality issues have been satisfactorily addressed during all project phases AND there is evidence of project staff actively advocating for gender equality on a regular basis.
3 Satisfactory Gender equality issues have been satisfactorily addressed during all project phases (project planning/preparation, implementation, monitoring and reporting/evaluation OR there is evidence of project staff actively advocating for gender equality on a regular basis.
2 Moderately Satisfactory Gender equality issues have been satisfactorily addressed in some of the project phases.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
52
Evaluation Directorate – SPPB, July 2010
Dimension Assessment Criteria Rating Rating Criteria
1 Unsatisfactory Gender equality issues have been satisfactorily addressed in one project phase only (e.g. during planning, but not during implementation).
0 Very unsatisfactory There is no evidence of gender equality issues having been satisfactorily addressed at any stage of the project.
5.2 Environment Environment issues were satisfactorily addressed during preliminary analysis conducted in preparation of the project.
Environment issues were satisfactorily addressed when engaging in policy dialogue with Malawian partners.
Environment issues were satisfactorily addressed during the project implementation process.
Environment issues were satisfactorily addressed in the monitoring and evaluation of project results achievement.
4. Highly satisfactory Environment issues have been satisfactorily addressed during all project phases AND there is evidence of project staff actively advocating for environment issues on a regular basis.
3 Satisfactory Environment issues have been satisfactorily addressed during all project phases (project planning/preparation, implementation, monitoring and reporting/evaluation OR: there is evidence of project staff actively advocating for environment issues on a regular basis.
2 Moderately Satisfactory Environment issues have been satisfactorily addressed in some of the project phases.
1 Unsatisfactory Environment issues have been satisfactorily addressed in one project phase only (e.g. during planning, but not during implementation).
0 Very unsatisfactory There is no evidence of Environment issues having been satisfactorily addressed at any stage of the project.
5.3 HIV/AIDS HIV/AIDS issues were satisfactorily addressed during preliminary analysis conducted in preparation of the project.
HIV/AIDS issues were satisfactorily addressed when
4. Highly satisfactory HIV/AIDS issues have been satisfactorily addressed during all project phases AND there is evidence of project staff actively advocating for HIV/AIDS issues on a regular basis.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
53
Evaluation Directorate – SPPB, July 2010
Dimension Assessment Criteria Rating Rating Criteria
engaging in policy dialogue with Malawian partners.
HIV/AIDS issues were satisfactorily addressed during the project implementation process.
HIV/AIDS issues were satisfactorily addressed in the monitoring and evaluation of project results achievement.
3 Satisfactory HIV/AIDS issues have been satisfactorily addressed during all project phases OR there is evidence of project staff actively advocating for HIV/AIDS issues on a regular basis.
2 Moderately Satisfactory HIV/AIDS issues have been satisfactorily addressed in some of the project phases.
1 Unsatisfactory HIV/AIDS issues have been satisfactorily addressed in one project phase only (e.g. during planning, but not during implementation).
0 Very unsatisfactory There is no evidence of HIV/AIDS issues having been satisfactorily addressed at any stage of the project.
5.4 Governance Governance issues were satisfactorily addressed during preliminary analysis conducted in preparation of the project.
Governance issues were satisfactorily addressed when engaging in policy dialogue with Malawian partners.
Governance issues were satisfactorily addressed during the project implementation process.
Governance issues were satisfactorily addressed in the monitoring and evaluation of project results achievement.
4. Highly satisfactory Governance issues have been satisfactorily addressed during all project phases AND there is evidence of project staff actively advocating for good governance on a regular basis.
3 Satisfactory Governance issues have been satisfactorily addressed during all project phases (project planning/preparation, implementation, monitoring and reporting/evaluation OR there is evidence of project staff actively advocating for good governance on a regular basis.
2 Moderately Satisfactory Governance issues have been satisfactorily addressed in some of the project phases.
1 Unsatisfactory Governance issues have been satisfactorily addressed in one project phase only (e.g. during planning, but not during implementation).
0 Very unsatisfactory There is no evidence of Governance issues having been satisfactorily addressed at any stage of the project.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
54
Evaluation Directorate – SPPB, July 2010
Dimension Assessment Criteria Rating Rating Criteria
6. Efficiency Financial and human resources managed efficiently. This can be demonstrated – for example – through:
Number and qualification of project staff adequate.
Adequate ratio of program administrative costs to program costs over time
Evidence of measures taken/strategies/processes used to ensure project efficiency.
4 Very satisfactory If project reports and/or consultations with stakeholders provided considerable evidence of attention to and efforts in realizing project efficiencies.
3 Satisfactory If project reports and/or consultations with stakeholders provided some evidence of attention to and efforts in realizing project efficiencies.
2 Moderately satisfactory If project reports and/or consultations with stakeholders provided a few examples of attention to and efforts in realizing project efficiencies.
1 Unsatisfactory If project reports and/or consultations with stakeholders provided no evidence of attention to and efforts in realizing project efficiencies.
7. CIDA Management Principles
Project Design and implementation have furthered country ownership of development results (e.g. by making explicit links to national sector strategies/plans).
Project design and implementation have allowed for alignment with Malawian partner systems (e.g. planning, financial, procurement).
Project design and implementation have demonstrated will for harmonization of development support with other donors (e.g. sharing information or resources, using common systems or approaches).
4. Highly satisfactory If all three criteria apply
3 Satisfactory If two criteria apply
2 Moderately satisfactory If one criterion applies
1 Unsatisfactory If none applies
8. RBM/ Monitoring & Evaluation (M&E)
Appropriate management systems and resources developed for managing the project for results according to CIDA guidelines.
Project has applied RBM principles and tools (including SMART results indicators) in meaningful ways throughout the project cycle.
Appropriate processes put into place and used to identify and mitigate project risks.
4. Highly satisfactory If all three criteria apply
3 Satisfactory If two criteria apply
2 Moderately Satisfactory If one criterion applies
1 Unsatisfactory If none applies
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
55
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx II VV SS tt aa kk ee hh oo ll dd ee rr ss CC oo nn ss uu ll tt ee dd Name Position Method of Consultation
CIDA Malawi Program (Bilateral) – CIDA HQ
Banwell, Anne Project Team Leader, Malawi Program, Geographic
Programs Branch, CIDA
Interview
Cudmore, Carolyn Former Program Manager, Malawi Program, Africa
Branch, CIDA
Phone Interview
Dowswell, Sarah Former Program Manager, Malawi Program, Africa
Branch, CIDA
Interview
Edmunds, Amanda Analyst, Integrated Planning and Accountability
Division, HRSDC, formerly Project Team Leader,
Malawi Program, Africa Branch CIDA
Interview
Miloff, Maury Former Program Manager, Malawi Program, Africa
Branch, CIDA
Interview
Paquette, Stefan Senior Development Officer, Malawi Program,
Geographic Programs Branch, CIDA
Interviews
Parsons, James Program Manager, Malawi Program, Geographic
Programs Branch, CIDA
Interviews
CIDA Malawi Program – Malawi Office of the CHC, PSU and Malawi-based
Foote, Robert PSU Director (since 2006), Malawi PSU Interviews, group meetings
Jere, McPherson CIDA Education Specialist, Malawi PSU Interview
Kacheche, Brenda Project Co-Manager, Gender Equality Support Project
(GESP), Malawi-based.
Group interviews, Site visits
Kampala, Robert Former Water & Sanitation Specialist, Malawi PSU
(now Country Director, WaterAid)
Interview
Killick, Peter Aid Liaison Officer, PSU Interview
Kumwenda, Wilson Project Field Manager, Government Assistance Project
(GAP), Malawi Based
Interview (in November
2008 only)
Lebede, Johannes Former Project Advisor, HIV/AIDS Project, , Malawi
based
Interview
Liwewe, Olivia Mchaju Former CIDA Gender Advisor, Malawi PSU, DAGG
Member
Interview
Malumelo, Roman Project Co-Manager, Gender Equality Support Project
(GESP), Malawi Based
Interview
Mambulu, James Water and Environment Specialist, Malawi PSU Interview, site visits
Nyirenda, Michael Project Director. Project on Economic Governance
(PEG), Malawi based,
Interview
Polepole, McLloyd
Mwayi
Project Field Manager, GSES II, Malawi based. Interview
Roome, Roger First Secretary (Development), O/CHC Individual and group
interviews
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
56
Evaluation Directorate – SPPB, July 2010
Name Position Method of Consultation
Sanjeni, Wictor Former Gender Specialist, Malawi PSU Interview
Young, Valerie Acting High Commissioner and Head of Cooperation,
Canadian High Commission in Malawi, O/CHC
Interviews
Zulu, Dorothy Former GESP Director, Malawi based Interview
Other staff in CIDA HQ
Beaulieu, Rémy Evaluation Manager, Evaluation Directorate, Strategic
Policy and Performance Branch, CIDA
Ongoing consultations
Clement, Louise Director General, Southern and Eastern Africa
Directorate, Geographic Programs Branch, CIDA
Interview
Cmoc, Natalka Team Leader - Results Based Management,
Performance Management Directorate, Strategic Policy
and Performance Branch, CIDA
Interview
Dicum, Julia Senior Analyst, Health and Education Directorate,
Sectors and Global Partnerships Branch, CIDA
Interview
Fortin, Marie Eve Development Officer, Senegal Program, formerly
Performance Review Officer, Strategic Policy and
Performance Branch, CIDA
Consultations
Fournier, Suzanne Project Manager, Multilateral and Global Programs
Branch, CIDA
Email consultation
Galvin, Melanie Acting Manager, Global Initiatives Directorate,
Multilateral and Global Programs Branch, CIDA
Email consultation
Guthrie, Tina Manager, Food Aid Unit, Multilateral and Global
Programs Branch, CIDA
Email consultation
Hébert, Maryse Evaluation Manager, Evaluation Directorate, Strategic
Policy and Performance Branch, CIDA
Ongoing consultations
Kostiuk, Nadia Director General
Canadian International Development Agency
Southern and Eastern Africa Directorate (BFO)
Geographic Programs Branch
Interview
MacCormack, Julie Senior Program Officer, Food Aid Unit, Multilateral
and Global Programs Branch, CIDA
Email consultation
Malenfant, Brigitte Senior Analyst, Sectors and Global Partnerships
Branch, CIDA
Group meetings
Manirabonara, Richard Program Policy Analyst, Canadian Partnership Branch,
CIDA
Group meetings
McLean, Erin Project Manager, Multilateral and Global Programs
Branch, CIDA
Email consultation
Mundle, Carol Acting Manager, Volunteer Cooperation, Canadian
Partnership Branch, CIDA
Email and phone
consultation
Neufeld, Ken Director, Mozambique Program, Geographic Programs
Branch, CIDA
Interview
Oke, David International Development Project Advisor, Canadian
Partnership Branch, CIDA
Email consultation
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
57
Evaluation Directorate – SPPB, July 2010
Name Position Method of Consultation
Payne, Leslie Project Manager, Multilateral and Global Programs
Branch, CIDA
Email consultation
Porter, Ken Project Manager, Multilateral and Global Programs
Branch, CIDA
Email consultation
Semeluk, Loree Development Officer, Pan Africa Program, Geographic
Programs Branch, CIDA
Email consultation
Singh, Goberdhan Director, Evaluation Directorate, CIDA Ongoing consultations
Thériault, Larry Evaluation Manager, Evaluation Directorate, CIDA Ongoing consultations
Turmel, Rémi Advisor, International Projects, Canadian Partnership
Branch, CIDA
Email consultation
High Commission/Embassy
Deyell, John (Retired). Head of Mission in Zambia, 2004-2007 Phone interview
Project Monitors/Evaluators19
Lalonde, Helene Monitor, GESP Interview
Ogilvie, Keith Project Monitor on Governance, and Transitional
Director, PEG (3 months in 2006)
Phone Interview
Ramsay, Ted Monitor, GAP and GSES Interview
Canadian/International Implementing Partners
Donaldson, Joan Project Director, GAP, Agriteam Canada Interview and email
consultation
Loga, Tiwonge Country Program Manager, CPHA (SAT Implementing
Agency)
Interview
Mapemba, Jacob P. Country Director/Agriculture and Rural Dev Specialist,
WUSC (CFA for GSES II20
& CPB partner)
Interview
Miller, Anna Director of Programs, PSU, CPAR (CPB and MGPB
partner)
Email consultations
Munyao, Kioko World Vision Canada Email consultation
Nychyperovych,
Tatyana
CARE Canada (CFA for GSES II & CPB partner) Interview
Paterson, Richard CARE Canada (CFA for GSES II & CPB partner) Interview
Sandström, Anita Executive Director, CPHA (SAT Implementing
Agency)
Email consultation
Syme, Hilary Former Program Officer, Malawi Program Africa
Branch, CIDA
Email consultation
19
Michael Miner was also contacted for an interview but the interview never transpired.
20 CARE and WUSC are part of a consortium facilitating the implementation of GSES II. The consortium initially
included GDG (Global Development Group) which has now been dissolved.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
58
Evaluation Directorate – SPPB, July 2010
Cross Sectoral (e.g. re Program level information, overall donor ‘landscape in Malawi, SWAps)
Chabwana, Bridget Director of Policy and Programs, National AIDS
Commission, Government of Malawi
Interview
Dr. Ngwira, Naomi Director, Debt and Aid Division, Ministry of Finance,
Government of Malawi
Interview
Dr. Phoya, Anna Ministry of Health, Government of Malawi Interview
Dr. Sangela, Former Principal Secretary Health, Ministry of Health,
Government of Malawi
Interview
Nkhata, Stan Ministry of Finance, Government of Malawi Interview
Schouten, Erik and
Director
Ministry of Health, Government of Malawi Interview
Kamlongera, Dr.
Augustine
Director of Planning, Ministry of Education, Science
and Technology, Government of Malawi
Interview
Zimpita, Patricia Director, Planning, OPC Interview
Education Sector
Agabu, Rafael Deputy Director, Education Methods & Advisory
Services, MoEST, Government of Malawi
Individual Interview
Banda, Charles Chief Procurement Officer, Procurement and Supplies
Unit, MoEST, Government of Malawi
Individual Interview
Botha, Anne Chairperson PCAR CC and Assistant Statistican,
MoEST, Government of Malawi
Individual Interview
Chazema, Mrs.
Jacqueline
Gender and HIV Coordinator, Domasi College Individual Interview
Chipungu, Alex Supplies Unit, MoEST, Government of Malawi Individual Interview
Chitimbe, Moffat Principal Secretary, MoEST, Government of Malawi Individual Interview
Chakwera, Dr. Elias Current Director, Domasi College Individual Interviews
Goertler, Dr. Wilfried Former GTZ representative in Malawi, Education
Donor Coordination Group
Individual interview
Dr Hau, Simeon Former Director, Domasi College Individual Interviews
Susewele, Dr. William Deputy Director, Malawi Institute of Education Individual interview
Fris, Jorgen GTZ, Basic Education Program Individual interview
Gunsaru, Charles Director, Malawi Institute of Education Individual interview
Iphani,Max Assistant Director, Malawi Institute of Education Individual
Kaambakadzanja, Dave PICAR coordination committee Interview
Kalanda, McKnight
Director Basic Education, MoEST, Government of
Malawi
Individual Interview
Kamlongera, Augustine Director of Planning, Ministry of Science and
Technology (MoEST), Government of Malawi
Individual Interview
Kaperemera, Nelson Former Acting Director, MIE and former Director
Domasi College
Individual Interviews
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
59
Evaluation Directorate – SPPB, July 2010
Khombe, Mrs. Gender Desk Officer, MoEST, Government of Malawi Individual Interview
Kulima, Mercy Procurement Officer, Procurement and Supplies Unit,
MoEST, Government of Malawi
Individual Interview
Magreta, Lonely Director of Secondary Education, MoEST,
Government of Malawi
Individual Interview
Mardes, Gorge GTZ Basic Education Program Individual Interview
Matale, Enock Assistant Statistician, MoEST, Government of Malawi Individual Interview
Mhango, Dr. Martin Coordinator for Centre for Distance Learning, Domasi
College
Individual Interview
Mphisa, Simon UNICEF, Chief Education and youth Development,
Education Donor Coordination Group
Individual interview
Nyirenda, Stanley Consultant under GSES II Individual Interview
Phiri, Amin Principal Procurement Officer, Procurement and
Supplies Unit, MoEST, Government of Malawi
Individual Interview
Polepole, McLloyd
Mwayi
Project Field Manager, GSES II Interview
Tsokonombwe,
Sylvester
Consultant under GSES II Individual Interview
Gender Equality
Chikakheni, Olivia Principal Secretary – Gender, Ministry of Women and
Child Development, Government of Malawi
Individual Interview
Kaliya, Ema Chair, Malawi NGO Gender network Individual Interview
Kassonga, Franck NATAFOGAI representative Group Interview
Matemba J., Marika NATAFOGAI representative Group Interview
Mihowa, Linga Program and Policy Manager, OXFAM Malawi and
DAGG member
Group Interview
Phiri, Sekaame NATAFOGAI representative Group Interview
Puwaruwa, Jessi NATAFOGAI representative Group Interview
Salima, Naile Gender Advisor, OXFAM Malawi (also provides
technical assistance to DAGG)
Group Interview
Chiwambe Program Officer – Thyolo District (manager of
GESP‟s subproject), OXFAM Malawi
Group Interview
Malawi Broadcasting
Corporation
Management Group
Director of Engineering, Control of Program, Control
of Research, Control of Administration
Group Interview
Health/ HIV/AIDS
Banda, Derio Associate Director, Ministry Quality Assurance
World Vision, Malawi
Group Interview
Barnhart, Matt In charge of HIV/AIDS, USAID Malawi Interview
Cameron, Alisa Head of Health Department, USAID Malawi Interview
Chabwana, Bridget Director of Policy and Programs, World Vision Malawi Group interview
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
60
Evaluation Directorate – SPPB, July 2010
Chipimo, Miriam Manager, Reproductive Health, HIV/AIDS, UNICEF Interview
Chitekwe, Stanley Project Officer, Nutrition, UNICEF Interview
de Cleene, Anna (Ex) DFID HIV AIDS Advisor Interview
Dorothy Namate Global Fund Local Agent, Price Waterhouse Cooper Interview
Dr. Loga, Tiwonge Malawi Country Programme Manager for AIDS Trust
Southern Africa, SAT Malawi
Interview
Dr. Mwale, Biziwick Executive Director, National AIDS Commission Group & individual
Interviews
Johns, Desmond Country Coordinator, UNAIDS Interview
Kaimvi, Washington Director, Finances and Administration, National AIDS
Commission
Group & individual
Interviews
Kalepa, Humphrey J. World Vision Malawi Group Interview
Kapyepye, Ethel Head of HIV/AIDS, World Vision Malawi Group Interview
Lane, Jason Team Leader – Health and HIV/AIDS, DFID Malawi Interview
Lazaro, Dorothy National Programme Officer, UNFPA Interview
Matonya, Sarah HIV/AIDS Advisor, DFID Malawi Interview
Namarika, Rose Senior Manager Heath and Nutrition, World vision
Malawi
Group Interview
Nyambose, Madalo Assistant Director, Debt and Aid Division, Ministry of
Finance, Government of Malawi. Chair of the of
HIV/AIDS pool and chair of the Finance committee of
the health SWAp
Interview
Seip, Ragnhild Second Secretary – Development, Royal Norwegian
Embassy
Interview
Thole, Tchaka Director Finance and Administration/Acting Country
Director, CPAR Malawi
Interview
Yilla, Mamadi Country Coordinator for the US Government AIDS
Program, PEPFAR
Interview
Zimba, Joseph Programme Manager, CPAR Malawi Interview
Water and Sanitation
Gondwe, Boniface N.C. Director, Water Supply and Sanitation, Ministry of
Water and Irrigation Development, Government of
Malawi
Interview
Kazombo, Joseph COMWASH Project Coordinator, Ministry of Water
and Irrigation Development, Government of Malawi
Group interview
Limphangwe Water
Scheme Management
Committee
COMWASH beneficiary communities Site visit/ Group Interviews
Malunga, Grain Former Principal Secretary, Water, Ministry of Water
and Irrigation Development, Government of Malawi
Interview
Mamba, Jeffrey Ministry of Water and Irrigation Development,
Government of Malawi
Group Interview
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
61
Evaluation Directorate – SPPB, July 2010
Migowi Water Scheme
Management
Committee (2
representatives)
COMWASH beneficiary communities Site visit/ Group Interviews
Members of the Water
and Sanitation
Development Partners
Group
Representatives from AFDB, JICA, UNICEF Group Interview and
meeting observation
Namizunga, George Former COMWASH Project Coordinator, COMWASH
Malawi
Individual interview
Crispin Former Phalombe District Water Management Officer
(now Community Based Management Resp., MoIWD
Regional Office, Blantyre)
Individual Interview
Governance
Bamusi, Mavuto Civil Society-Human Rights Consultative
Committee/Malawi Economic Justice Network
Focus Group/ Group
Interview
Chikoko, Mulle DFID Group interview
Chunga, Dixon Director, Planning, DHRMD Interview
Forster, Stuart DFID Group interview
Hon. Chimango, Louis Member of Parliament, Government of Malawi Group Interview
Kawaye, Jimmy Governance Advisor, DFID Group interview
Kumbatira, Andrew Civil Society-Malawi Economic Justice Network Focus Group/ Group
Interview
Kwataine, Martha Civil Society-Malawi Health Equity Network Focus Group/ Group
Interview
Magalasi, Collins Civil Society-Malawi Economic Justice Network Focus Group/ Group
Interview
Mazengera, Hennox Civil Society-Society of Accountants in Malawi Focus Group/ Group
Interview
Mhoni, Victor Civil Society-Civil Society Agriculture Network Focus Group/ Group
Interview
Mwanyungwe, Abel Media – Polytechnic Focus Group/Group
Interview
Njolomole, Henry Member of Parliament, Government of Malawi Group Interview
Semphere, Patrick Media-TransWorld Radio Focus Group/Group
Interview
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
62
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx VV DD oo cc uu mm ee nn tt ss RR ee vv ii ee ww ee dd
CIDA Documents Program or Agency Level
“Malawi Country Profile”. CIDA (2008)
“Canada-Malawi Relations”. DFAIT (2009)
“Malawi Development Trends”. CIDA (2006)
“Malawi Country Program Delivery Strategy 1998-2008”. CIDA (1997)“
“Revised Malawi Logical Framework”. CIDA (2003)
“Program Performance Report 2006-2007”. CIDA (2007)
“Program Performance Report 2005-2006”. CIDA (2006)
“Malawi Donor Ranking”. CIDA (April 2007)
“Mid-Term Country Program Review of the CIDA Malawi Program, 1999-2009”. Michael Miner
for CIDA (November 2004)
“Report on Plans and Priorities 2006-2007 (DRAFT)”. CIDA, 2006
“Report on CIDA‟s Social Development Priorities for Malawi”. CIDA (2006)
“Results Based Management Policy Statement”. CIDA (2008)
“CIDA‟s Policy on Gender Equality”. CIDA (1999)
“Draft Guide for Preparing a Country Program Development Framework”. CIDA (2002)
Government of Malawi
“Malawi Vision 2020”. Malawi Sustainable Development Network Program, Government of
Malawi (2003-2005) < http://www.sdnp.org.mw/~esaias/ettah/vision-2020/ >
“Malawi Growth and Development Strategy 2006-2011”. Ministry of Finance, Government of
Malawi (2005)
“Malawi Growth and Development Strategy: Annual Review 2006/07 Year, Synthesis Report”.
Ministry of Economic Planning and Development, Government of Malawi (2007)
“Implementing the Development Assistance Strategy (DAS): Institutionalizing Sector
Coordinating Groups”. Ministry of Finance, Government of Malawi (2008)
“MGDS Annual Review 2006-2007: Synthesis Report on Progress Made in Implementing the
MDGs”. Ministry of Economic Planning and Development, Government of Malawi (2008)
“Malawi Poverty Reduction Strategy”. Government of Malawi (August 2006)
“Effective Technical Cooperation for Capacity Development – Malawi Country Case Study”.
United Nations Development Programme and Office of the President, Government of Malawi
(2008)
“Drivers of Change and Development in Malawi – Working Paper”. David Booth et al., Overseas
Development Institute (January 2006)
“Malawi Poverty Reduction Strategy: Third Annual Progress Report and Joint IDA–IMF Staff
Advisory Note”. International Development Association and International Monetary Fund (2006)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
63
Evaluation Directorate – SPPB, July 2010
Other Donors Evaluations/Documents
“Country Brief and Data Profile: Malawi”. World Bank (2009)
“Country Brief and Data Profile: Malawi”. UNESCO Institute of Statistics (2006)
“Human Development Indices”. Human Development Report 2007/08. World Bank (2008)
“Malawi Millennium Development Goals Report 2008”. United Nations Development
Programme (2008)
“Joint Evaluation of the UNDG Contribution to the Paris Declaration on Aid Effectiveness”.
IFAD, UNAIDS, UNECA, UNIFEM, UNFPA, UNDP (2008)
“Development Assistance Strategy: Malawi”. United Nations Development Programme (2006)
“Aid Architecture: An overview of the main trends in Official Development Assistance flows”.
International Development Association (February 2007)
“Harmonizing Donor Practices for Effective Aid Delivery, Vol. 2: Budget Support, SWAps and
Capacity Development in Public Finance Management”. OECD-DAC (2006)
“Glossary of Key Terms in Evaluation and Results Based Management”. OECD-DAC (2002)
“Malawi Country Program”. Oneworld.net (March 2008)
“Report: Experts Meeting on Vertical Funds”. Eurodad (June 2008)
“Effective Aid by 2010? What it will take: Vol. 2 Country Chapters, Malawi”. 2008 Survey on
Monitoring the Paris Declaration. 3rd High Level Forum on Aid Effectiveness (September 2-4,
2008, Accra, Ghana)
“Evaluation of DfID‟s Country Programmes: Malawi 2000-2005”. Chris Barnett et al.,
Department for International Development (DfID) (April 2006)
“From Projects to SWAPs: An evaluation of British Aid to primary schooling 1988-2001”. DfID
(September 2002).
“Evaluation of General Budget Support – Malawi Country Report 1994-2004”. Jens Claussen et
al., University of Birmingham (April 2006)
“General Budget Support and Sector Program Assistance in Malawi”. United States Agency for
International Development (USAID) (November 2004)
“What Conditions favour the success of General Budget Support and Sector Program Assistance?
Malawi Country Case Study”. USAID (2004)
“Malawi Country Assistance Evaluation”. World Bank (July 2006)
“Enhanced Heavily Indebted Poor Countries (HIPC) Debt Initiative: Completion Point
Document, Malawi” World Bank (August 2006)
“African Economic Outlook: Malawi”, Organization for Economic Cooperation and
Development (2005)
“Canadian Volunteer Engagement Program 2002-2005: Interim Narrative and Financial Report”.
World University Service of Canada (WUSC) (September 2003)
“Project/Program Performance Report: WUSC 2002-2004 Development Program”. Donna M.
Shawartzburg for WUSC (2004)
“Canadian Volunteer Engagement Program 2001-2002: Semi-Annual Report, April-September”.
WUSC (November 2003)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
64
Evaluation Directorate – SPPB, July 2010
“Joint Donor Evaluation. Managing Aid Exit and Transformation .Malawi Country Case Study”.
AAVV (May 2008).
“Joint Donor Evaluation. Managing Aid Exit and Transformation. Lessons from Botswana,
Eritrea, India, Malawi and South Africa. Synthesis Report”. Slob & Jerve (May 2008)
“Managing aid – Practices of DAC member countries”. OECD-DAC (June, 2009)
Literature on PBAs
“Chapter 3: Sector Wide Approaches.” Harmonizing Donor Practices for Effective Aid Delivery.
OECD (2006)
“Primer on Project Based Approaches”. Policy Branch, CIDA (August 2003)
“Capacity Development under PBAs”. Results from the LENPA Forum (April 2005)
Survey results on SWAPs and SWAp like Initiatives in CIDA – Case of HIV/AIDS in Malawi
“Gender equality in Program Based Approaches: Experiences, Lessons learned and Good
practices”. CIDA (2005)
“Chapter 7: Core Funding, Section 7.4: Program Based Approaches”. Business Process
Roadmap. CIDA CIDA‟s Business RoadMap (2009)
“Accountability Issues in Program Based Approaches”. L.M. Edelsward for the Portfolio
Initiative (2003)
“Operational Guide on Direct Budget Support and Pooled Funding to Recipient Countries”.
Human Resources and Corporate Services Division, CIDA (May 2005)
Documents on specific CIDA Projects and on respective Sectors
Education
“Malawi Education Sector Integrated Risk Assessment”. Geneviève Brown et al. for CIDA (May
2007)
“National Education Sector Plan, 2008-2017”. Ministry of Education, Science and Technology,
Government of Malawi (2008)
“Education Sector Policy and Investment Framework 2000-2015”. Ministry of Education,
Science and Technology, Government of Malawi (2002)
“Improving the Quality of Primary Education in Malawi: Project Approval Document”. CIDA
(2007)
“Enhancing the Quality of Primary Education in Malawi: Final Annual Report”. UNICEF (March
2008)
GSES II:
– Project Approval Document
– Project Implementation Plan
– Memorandum of Understanding
– Annual Workplans
– Semi-Annual Reports
– Mid Term Review Report (2008)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
65
Evaluation Directorate – SPPB, July 2010
SSTEP:
– Impact Evaluation Final Report (2008) and Terms of Reference
– Final project report (2007)
GTZ:
– Administrative Arrangement between the Government of Canada and GTZ GMBH
– Email communication between Stefan Paquette and Mades Georg (October 21, 2008)
– GTZ Website (www.gtz.de)
Health including HIV/AIDS
“Malawi National HIV/AIDS Policy”. Government of Malawi (2003)
Implementation of the Malawi HIV&AIDS Strategic Management Plan (2003 – 2008)/ Mid-term
evaluation (December 2006)
“Malawi‟s HIV/AIDS Mini-SWAp: A case study of risk management and capacity building
under the program approach”. (2002)
“Malawi Health SWAp, Mid Term Review Summary Report”, NORAD, (January 2008)
“Malawi Micronutrient and Health (MICAH) Program Phase II Final Report (Including FY 06):
November 2002-October 2005”. World Vision Malawi (2005)
“Improving Nutrition of Women and Children: The MICAH Program, Final Program Report”.
World Vision (2006)
“MICAH FY 04 Annual Report: October 2003-September 2004”, p. 42. World Vision Canada
(January 2005)
“MICAH Malawi End of Phase I Report”. World Vision Malawi
“Annex A: Preventing and Mitigating the Impacts of HIV/AIDS in Malawi, Canadian Volunteer
Engagement Program 2001-2002: Final Report”. World University Service of Canada (February
2003)
HIV Pooled Fund:
– Project Approval Document
– Memorandum of Understanding
– High Commission letters
– “HIV/AIDS Action Framework 2007-2001: PAD for Pooled Donors”
– Pooled Funding Partner meeting documents
– HIV/AIDS Development Partner Group meeting documents
– National AIDS Commission related documents
– Annual and weekly reports
– Project Performance Reports (04/05 - 07/08)
– Assessment of impact of national response
– Implementation of the Malawi HIV&AIDS strategic management plan (2003 – 2008) Mid-
term evaluation
– HIV/AIDS Baseline Sector Survey.
SAT:
– Mid-Term Performance Review
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
66
Evaluation Directorate – SPPB, July 2010
– Completion report
– Annual reports
– “SAT‟s work with the „Three Ones‟, Malawi‟s experience”
CPAR:
– Project Implementation Plan
– Semi-Annual reports
– End of Project reports
– Email communication between Eric Bellows and Sarah Dowswell (April 15, 2002)
– Program Evaluation of CPAR‟s „Moving Beyond Hunger‟ and „Reducing the Burden of
HIV/AIDS in Rural Malawi‟ programs
Water and Sanitation
“Sector Financing: Water Supply and Sanitation in Malawi”. Malawi Economic Justice Network
and Water Aid (2006)
COMWASH:
– Preliminary Project Design & Project Management Strategy
– Project Implementation Plan
– Project Approval Document
– Memoranda of Understanding
– Quarterly and annual reports
– Monitoring reports
– Completion Report
– Mid Term Evaluation
Governance
GAP:
– Memorandum for the Minister
– Project Implementation Plan
– Project Approval Document
– Memorandum of Understanding
– Mid Term Evaluation
– Annual workplans
– Quarterly, semi-annual and annual reports and stakeholder comments
– Annual Output to Purpose Review
– Monitoring Reports and monitoring mission debrief summary
PEG:
– Project Implementation Plan
– Project Approval Document
– Annual workplans
– Quarterly and annual reports
– Monitoring reports and monitoring mission debrief summary
– Job Descriptions
– Project Advisory Meeting minutes
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
67
Evaluation Directorate – SPPB, July 2010
– Sample of Sub-project Approval documents
– Sample of Terms of Reference
– Sample of special studies.
PEG & GESP:
– Memorandum of Understanding with DfID
– Annual reports to DfID.
Gender
“Gender Needs Assessment of the MGDS”. Olivia Liwewe, Dr. Naomi Ngwira, Bright Sibale
(July 2006)
“Revised Gender Policy”, Government of Malawi (2008)
“National Gender Program 2004-2009”. Ministry of Gender, Child Welfare and Community
Services, Malawi.
“Managing Human resources within the Government of Malawi – Guidelines on Gender
Mainstreaming”. Government of Malawi (2007)
GESP:
– Concept Paper
– Project Approval Document
– Project Implementation Plan
– Memorandum of Understanding
– Project Advisory Committee minutes
– Quarterly and annual Reports
– Monitoring Reports
– Final Evaluation report
– Annual Workplans
– Research report on “Women and Men in Decision Making Positions” (2008)
PSU
PSU
– Project Approval Document
– Memoranda of Understanding
– Annual Workplans
– Semi-annual and annual reports
– Audit reports
– Sector Specialist Job descriptions
Other Documents
“Bank Crisis Impact Limited in Africa – AfDB Economist”. Reuters Interview (October 2008)
“Free Primary Education Backfires”. Brian Ligometa, News From Africa (2002)
Evaluation Methods for Studying Policies and Programs. Carol Weiss, Prentice Hall (1998)
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
68
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx VV II PP rr oo jj ee cc tt ss RR ee vv ii ee ww ee dd
Project Number Title Total Budget
(in Millions)
Sector
Bilateral Investments
A030418001 HIV/AIDS Program Support (HIV AIDS) $14.8 Health
A020397001 Community Water, Sanitation & Health (COMWASH)
$12.7 Health
A031403001 Grant Support to the Education Sector Phase II (GSES II)
$20.0 Education
A020401001 Secondary School Teacher Education Project (SSTEP)
$6.7 Education
A030571001 Project for Economic Governance (PEG) $3.6 Governance
A020392001 Government Assistance Project (GAP) $7.8 Governance
A030927001 Gender Equality Support Project (GESP) $2.9 Gender Equality
*A020576001
*A031056001
*A031484001
*A033764-001
Program Support Unit (PSU) 21
Program Support Unit Phase II
Program Support Unit Phase III
Program Support Unit Phase IV
$12.7
Multi-Bi and Bi-Bi Projects
A030608-001 Improving the Quality of Primary Education Project (UNICEF)
$7.5 Education
A033400-001 Initial Primary Teacher Education (GTZ) $5.0 Education
CPB and Multilateral Investments
S062092001 (CPB) CPAR-CAW HIV/AIDS Project-Malawi 2002-05 $1.3 Health
M003861003 (Multilateral)
CPAR – MALAWI $1.6 Health
S061235PRG (CPB) WUSC – Program Support For 2001/2002 $1.3 Multi-sector
S061999 (CPB) WUSC 2002 - 2004 Development Program $1.9 Multi-sector
M003861002 (Multilateral)
World Vision Canada – Malawi $1.4 Health
M003544 and M011053 (Multilateral)
AFRICA/SUB-SAHARA/MICRONUT/HEALTH
Micronutrients and Health MICAH Ph 1 & II, World Vision
$17.022
Health
21
These four projects were reviewed with a focus on their implications for overall program management issues (i.e.,
they were not reviewed against the same criteria used for the other projects).
22 This is a multi-country project and 37% of its budget went to Malawi.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
69
Evaluation Directorate – SPPB, July 2010
Project Number Title Total Budget
(in Millions)
Sector
M003979 (Multilateral)
Contribution to the World Food Program for the Women‟s Health and Micronutrient Facility (WHMF) in Zambia and Malawi 23
$10.7 24
Health
Africa Branch – Regional
A018903001 Developing Community Competence for HIV/AIDS in Southern Arica, Ph3 , , Canadian Public Health Association (CPHA)
$6.0 25
Health
Total $135.2
23
While this project was included in the originally envisaged sample we were unable to review it due to lack of
documents or contact persons who could have provided information on the initiative.
24 This is a multi-country project and 71% of its budget went to Malawi.
25 This is a regional project and 20% of its budget went to Malawi.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
70
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx VV II II MM aa ll aa ww ii CC PP EE PP rr oo jj ee cc tt RR aa tt ii nn gg ss The table below compares reviewed projects across criteria (see Annex I of the Technical Report for detailed project ratings). This information is
reflected throughout section 6 of the report. Highlights of the analysis are summarized below
The table provides a summary of the average ratings of the projects reviewed for the Malawi CPE and shows some interesting trends:
Projects were generally assessed as satisfactory to highly satisfactory in relation to Project Relevance, CIDA Management Principles, and
RBM/M&E – regardless of the sector.
In every sector but governance, projects were generally assessed as quite effective in realizing their results.
Most projects did very well in addressing gender equality issues, but were much less successful in addressing other cross cutting issues.
Projects had mixed ratings across sectors in relation to sustainability and coherence.
Dimension HIV/AIDS COMWASH GSES II SSTEP PEG GAP GESP Average
Rating by Criteria
Relevance Highly
Satisfactory (4) Satisfactory
(3) Highly
Satisfactory (4) Highly
Satisfactory (4)
Highly Satisfactory
(4)
Highly Satisfactory
(4)
Highly Satisfactory (4)
3.9
Effectiveness Highly
Satisfactory (4) Satisfactory
(3) Highly
Satisfactory (4) Highly
Satisfactory (4) Satisfactory
(3)
Moderately Satisfactory
(2)
Highly Satisfactory (4)
3.4
Sustainability Satisfactory (3) Moderately Satisfactory
(2) Satisfactory (3)
Highly Satisfactory (4)
Satisfactory (3)
Unsatisfactory (1)
Moderately Satisfactory (2)
2.57
Coherence Moderately
Satisfactory (2)
Moderately Satisfactory
(2) Satisfactory (3) Satisfactory (3)
Satisfactory (3)
Unsatisfactory (1)
Satisfactory (3) 2.4
Cross26
Cutting Issues
2 1.8 2.5 2.5 4 3.5 2 2.6
26
This was calculated by adding the average scores for each of the 4 cross-cutting issues divided by the total number of cross-cutting issues rated.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
71
Evaluation Directorate – SPPB, July 2010
Dimension HIV/AIDS COMWASH GSES II SSTEP PEG GAP GESP Average
Rating by Criteria
Cross Cutting Issues: Gender
Highly Satisfactory (4)
Moderately Satisfactory
(2)
Highly Satisfactory (4)
Highly Satisfactory (4)
Highly Satisfactory
(4)
Highly Satisfactory
(4) Not Applicable
3.7
Cross Cutting Issues:
Environment
Highly Unsatisfactory
(0)
Moderately Satisfactory
(2)
Highly Unsatisfactory
(0)
Highly Unsatisfactory
(0) Not Rated Not Rated
Highly Unsatisfactory (0)
.4
Cross Cutting Issues:
HIV/AIDS Not Applicable
Unsatisfactory (1)
Satisfactory (3) Satisfactory (3) Not Rated Satisfactory
(3) Satisfactory (3)
2.6
Cross Cutting Issues:
Governance
Moderately Satisfactory (2)
Moderately Satisfactory
(2) Satisfactory (3) Satisfactory (3)
Not Applicable
Not Applicable
Satisfactory (3) 2.6
Efficiency Not Rated
Between Moderately Satisfactory
and Unsatisfactory
(1.5)
Highly Satisfactory (4)
Not Rated Moderately Satisfactory
(2) Not Rated Satisfactory (3)
2.6
CIDA Mgmt Principles
Highly Satisfactory (4)
Moderately Satisfactory
(2)
Highly Satisfactory (4)
Highly Satisfactory (4)
Highly Satisfactory
(4)
Moderately Satisfactory
(2)
Highly Satisfactory (4)
3.4
RBM/M&E Highly
Satisfactory (4)
Highly Satisfactory
(4)
Highly Satisfactory (4)
Highly Satisfactory (4)
Highly Satisfactory
(4)
Highly Satisfactory
(4)
Highly Satisfactory (4)
4
Average Rating by
Project 3.3 2.4 3.6 3.6 3.4 2.5 3.3
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
72
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx VV II II II SS tt aa tt uu ss oo ff tt hh ee
RR ee cc oo mm mm ee nn dd aa tt ii oo nn ss oo ff tt hh ee MM aa ll aa ww ii CC oo uu nn tt rr yy
PP rr oo gg rr aa mm MM ii dd -- tt ee rr mm RR ee vv ii ee ww (( 22 00 00 44 ))
Recommendation Status
Program Level
1. Maintain Focus and Maximize Leadership: CIDA should continue to be
proactive as a strategic donor in its selected niches and maintain its present four core programming areas and focus on cross cutting themes.
Partially (50%) addressed
Until 2004, CIDA is reported to have an influential and highly respected role in Malawi for both its financial and non-financial support. CIDA made very few new investments after 2004.
The Malawi Program did its best to maintain its leadership in several sectors, within a very difficult context (prolonged uncertainty about the future of the program and no new money). For example, CIDA‟s leadership role in Gender was kept high by its engagement in the DAGG (no major investment but technical support). In 2009 CIDA was still viewed as a leader in GE, via GESP and its GE advisor in PSU. CIDA was also viewed as a strategic donor in the Water sector which benefited from its involvement in Sector coordination and the Planning Phase of the Water and Sanitation SWAp (including as coordinating agency for the sector). CIDA in Malawi played a lead role in HIV-AIDS right up to 2008, despite that it provided no new funding.
Despite this recognition, CIDA‟s reputation started to wane following 2006 due to its limited ability to invest in areas it deemed important.
2. Concentrated Focus of Projects:
Bilateral projects need to be in areas of concentration. Each present bilateral project should be completed and evaluated to ascertain return on investment.
Mostly (75%) addressed
As mentioned above, CIDA made very few new investments after 2004. In 2008, CIDA approved two new education sector projects (the $ 5 million “Initial Primary Teacher Education” Project and the $7.5 million “Improving the Quality of Primary Education” project).
Most of CIDA‟s bilateral projects were completed by the end of 2008. Some of them received no cost extensions and are currently in the process of being completed. The last ones should be closed by 2010.
Most CIDA bilateral projects were evaluated at mid term or at completion. However, limited attention was generally paid in these evaluation to the return on investment
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
73
Evaluation Directorate – SPPB, July 2010
Recommendation Status
3. Maintain Alignment with Priorities:
The CIDA Malawi Program needs to continue to consolidate its achievements and monitor how well it remains aligned to contextual changes within Malawi, CIDA, and internationally.
Partially addressed (50%)
Between 2004 and 2006, those responsible for the Malawi program were focused on keeping the program operational in the context of internal challenges (see main report Sections 3 and 9). Considerable attention was paid to ensuring that individual projects continued to be relevant, and changes were made to increase their relevance and/or appropriateness as required. For example, GAP was entirely revised and restructured in 2006 and COMWASH was refocused in 2005 following a very critical Midterm evaluation. In 2007, PEG was revised to become more responsive upon collapse of its main civil society partner (and as a result four organisations are now much stronger). Finally, GESP was refocused to move efforts from the social change level up to policy reform.
The situation was a bit different at the country program level. The CPDS was not updated to reflect changes in GoM or CIDA priorities. The need for doing so had reduced importance after 2006, due to changing CIDA needs and priorities. Between 2005 and 2008 the program team developed a number of plans and strategies that were not implemented because of the uncertainty related to the future of the program.
As noted in Section 9, there was little formal consolidation, monitoring and reporting at the program level.
4. Strategic Leveraging of Resources:
CIDA must remain a strategic player involved in the PRSP, SWAps, mini-SWAps, sectoral donor/government committees and well positioned in donor coordination groups.
Fully Addressed 100%
CIDA was actively involved with the Government and other interested donors in sector committees and donor coordination groups (e.g. the DAGG). It was further actively engaged in planning and preparing for the emerging SWAps in Water and Sanitation, and Education.
In all sectors stakeholders acknowledged CIDA‟s technical contributions to these processes, yet also noted that CIDA‟s inability to commit financial resources (e.g. to SWAps) had widely been noted with disappointment.
5. Gender Equality Lens: Gender should
be maintained as the key lens, and gender mainstreaming should be emphasized in the next five years.
Mostly addressed 75%
Those outside of CIDA (e.g. donors) lauded CIDA‟s role as a key player in relation to gender concerns. GE remained a core analytic point for some projects, such as PEG and GAP, and to some extent HIV-AIDS. Interviewed staff and project managers closer to the program indicate that attention to gender across individual investments became less systematic following the inception of the program and to some extent “optional” in some projects.
Management Level
6. Revise Program LFA and PMF: The
personnel in the CIDA HQ and PSU Teams should meet to continue the process of revising the Program level LFA and PMF and use these instruments as part of the process of measuring success on an ongoing basis. This is particularly important considering high staff turnover.
Marginally addressed 25%
While a revised version of the LFA was being developed, this document was never formally approved or used by program staff.
7. Consistency in Human Resources at the PSU: To prevent misperceptions and
misunderstandings, improve morale and reduce the likelihood of staff turnover, consistency is essential in PSU human resources and conducting performance appraisals.
Mostly addressed 75%
After 2005, various steps were taken to standardize HR processes in the PSU.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
74
Evaluation Directorate – SPPB, July 2010
Recommendation Status
8. Improve Cross-cultural and Cross-sectoral Communications:
Communication needs to increase through monthly meetings including cultural content, sharing new initiatives openly to enhance leveraging opportunities, and establishing a clearinghouse of information.
Mostly addressed 75%
At least since 2006 all PSU and High Commission staff met regularly (biweekly) for information meetings. Individuals reported that these meetings were most useful for information sharing purposes and – in a few cases - for identifying possible synergies on specific activities.
Those interviewed felt that the meetings could have been used more productively.
9. Staffing and Monitoring of the Malawi Program: Another desk officer should be
added to the program to allow officers to share administrative work and spend more time supporting the field. Considering the likelihood that CIDA staffing will not be increased and resources will decrease, proactive project monitoring is essential.
Not addressed 0%
No new positions were created. Moreover, there was considerable staff turnover in the program after 2004.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
75
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx II XX SS tt rr ee nn gg tt hh ss aa nn dd WW ee aa kk nn ee ss ss ee ss oo ff DD ee ll ii vv ee rr yy
MM oo dd ee ll ss
Type/Example Strengths Weaknesses Comments/Observations
Projects
General Flexible response mechanism, moderate preparation or negotiation time (6-12 months)
Responds to specific needs in a particular sector
Provides donors with high level of control over inputs – positive in view of need for accountability to home constituencies
Not reliant on local capacities and resources – therefore able to work faster, effectively and efficiently
Potential for disjointed and/or overlapping donor investments
Frequently donor rather than locally led, controlled, owned
In case of exit or phasing out by a donor , projects can pose more problems in terms of continuity: other donors will tend to be reluctant to move in, mainly because they have already committed most of their funds to other purposes.
Well known, familiar model
Directive versus Responsive projects
Directive projects Can focus on areas that in which CIDA has experience and expertise
Potentially in conflict with Paris Declaration principle of local ownership
How responsive a project is varies considerably depending on the staff involved in managing it, and the actual implementation approaches taken. (e.g., a formally „directive‟ project can be implemented in a highly participatory manner, while the opposite can be the case in a formally „responsive‟ one.
Responsive projects In line with Paris Declaration
Enhance likelihood of local ownership
Differences in Project Management
CEA led projects
(e.g. GAP, COMWASH, SSTEP)
Tend to manage effectively and efficiently
Likely to produce deliverables such as workplans and report on time and in acceptable format
Allows more direct interaction between Canadian Executing Agency (CEA) and CIDA HQ
May be very concerned about timely completion of outlined activities – can negatively influence willingness to take risks, engage in processes that require time (e.g. capacity development)
Less likely to build local capacity in project management
Consulted stakeholders noted considerable differences in the quality of work delivered by different CEAs. Being Canadian alone is not a guarantee for delivering good work.
CFA led projects Puts emphasis on capacity development both in thematic areas addressed and with
Time and work intensive for both CIDA and The only example in our sample (GSESII) was a positive experience. Consulted
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
76
Evaluation Directorate – SPPB, July 2010
Type/Example Strengths Weaknesses Comments/Observations
(GSES II) regard to project management and accountability
Can enhance likelihood of creating local ownership as key decisions are taken by local government (or other local players)
CFA
Can be risky for CFA as it is accountable for results but not in a position to make decisions crucial for project progress (only advises/recommends to local partners)
stakeholders pointed out however that the success of the project was largely due to exceptional staff both in Canada and Malawi, and also to the willingness and ability of CFA and CIDA officers to take risks and try out new approaches/trust local partners to make decisions.
Projects managed by local staff
(e.g. PEG, GESP)
In-depth knowledge of local context and stakeholders/players in the sector(s)
Contributes to local capacity/leadership development
Symbolic value of indicating CIDA‟s willingness to trust local leadership
(In some cases) lower management costs due to lower staff costs (see comment).
In relatively small country like Malawi with limited numbers of skilled individuals in different sectors where everyone tends to know everyone else, it can be challenging for local staff members to represent a Canadian funded project. In many cases staff have to consider their future beyond the project – and thus need to think about keeping connections/networks with local organizations/players intact.
In some cases, having local managers may require upfront capacity development and guidance from CIDA.
Unless managed carefully, there is the potential for perceived conflict of interest if the same PTL oversees and guides the implementation of such projects.
Management costs not always lower – in Malawi good people are sought after and are highly paid.
The in-kind support of PTLs needs to be taken into account.
Funds
(e.g. GESP)
Flexible
Can support/work with a variety of local organizations/beneficiaries and their respective areas of focus. Broad thematic spread.
Demand and needs driven – based on skills, priorities of local organizations/partners
Grants provided based on proposals submitted by local organizations can help build capacity both in thematic area, and related to proposal writing and financial management.
Short term agreements can be most appropriate in contexts where organizations have weak capacities and might not be able to commit to or
Short term support for broad variety of stakeholders with limited opportunity for follow up/long term engagement and limited capacity development
Fragmentation due to multitude of individual small investments
Less targeted on one specific area of intervention – Makes definition of objectives and monitoring of results more difficult
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
77
Evaluation Directorate – SPPB, July 2010
Type/Example Strengths Weaknesses Comments/Observations
conceptualize longer term collaboration or strategies
Program Based Approach
Malawi HIV/AIDS SWAp
Reduces duplication and overlap between individual investments
Focuses donor efforts on sector policies and institutional issues (depends on the capacity of the government; donor partners may focus on implementation)
Allows for better coordination and implementation of national response to HIV/AIDS through clearly identified institution (NAC)
Can reduce GoM transaction costs in the mid to long term. Increases GoM control. Promotes GoM accountability
Increases financial predictability at the sector level
Uses country systems and helps to build government capacity critical for sustainability
Likely to be able to absorb the exit of one donor if there are many donors contributing to the common basket and if the recipient is in control of the SWAp. Even so, the volume of support being withdrawn does make a difference.
Takes considerable time and investments to prepare/develop until more or less functional
Tends to be oriented to central government rather than local entities
Still in infancy and lessons being learned; stakeholders and the GoM are still adjusting
In the short term, tends to create new transaction costs for GoM and donors (e.g. for building consensus)
Not a panacea.
Requires new skills and abilities within GoM and among donors (e.g., donors often need to better understand public financial management and accounting to participate well and support a SWAp)
Question what role „big‟ players such as the Global Fund are taking on, and what implications their participation in a SWAp has for relevance of other players. In Malawi, the Global Fund provides largest share of overall SWAp, and has imposed numerous accountability mechanisms/procedures on the NAC.
Given that the Global Fund is funded by donors (including CIDA), raises the question of why donors decide to also provide bilateral funding to SWAp.
The binary distinction „project versus program‟ may be misleading. Some project type models (e.g., use of a CFA) share some key characteristics with a PBA – i.e. they give a lot of responsibility and decision making power to local partners. However, in PBAs the degree to which local government partners are free to make decisions about the use of donor resources can differ widely, too. It is not accurate to say that PBAs are always more „hands off‟ and less directive than projects.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
78
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx XX LL ii ss tt oo ff FF ii nn dd ii nn gg ss
Finding 1: CIDA‟s Malawi Country Program was relevant to the development and poverty reduction
priorities of the GoM, to Malawian priorities, to the evolving priorities of the Canadian
government and sectors of CIDA focus, and was complementary to the work of other
donors.
Finding 2: There is considerable evidence of positive achievements in all of CIDA‟s explicit and
implicit priority sectors in Malawi. However, overall effects tend to be project- rather than
sector-oriented, reflecting CIDA program management practices in Malawi.
Finding 3: In the health sector, the CIDA Malawi Program has made considerable contributions at both
the project and the sub-sector levels in the areas of HIV/AIDS and water and sanitation.
Given its focus on these sub-sectors, CIDA‟s impact on the Malawi health sector as a whole
has been modest.
Finding 4: In the education sector, CIDA has made significant contributions, particularly in the areas of
textbook supply and procurement and in secondary teacher education.
Finding 5: In the governance sector, CIDA helped to strengthen the roles played by civil society, the
National Assembly, and individual Members of Parliament in the national budget-making
process in Malawi. CIDA support has had modest effects on targeted Malawi government
departments.
Finding 6: In the gender equality sector, CIDA‟s contributions in Malawi are widely regarded as
remarkable, both in terms of project outputs and in CIDA‟s role as a leader and advocate for
gender equality in Malawi.
Finding 7: While the CIDA Malawi Program established a positive Canadian reputation among a broad
range of stakeholders in the country over the past decade, since 2006 the uncertainty related
to the future of the program has negatively affected Canada‟s reputation.
Finding 8: While CIDA support has contributed to an impressive number of results in Malawi since
1998, the likelihood that these will be sustained is quite mixed and modest overall. This
reflects Malawi‟s challenging context as well as the varied suitability of CIDA practices in
managing for sustainable results.
Finding 9: While CIDA‟s country program managers made some efforts to build coherence across
bilateral projects in Malawi, connections between bilateral projects have been largely
accidental. Some efforts have also been made to link bilateral priorities with CPB and
MGPB projects. This reflects the fact that the CPDS is a not a truly corporate document that
is binding on non-bilateral channels and that coherence among CIDA program investments
was not an over-riding CIDA priority during the period under review.
Finding 10: Bilateral projects within the Malawi Country Program have a mix of efficiency ratings.
Finding 11: According to established CIDA proxy measures for program efficiency, the Malawi Program
is relatively efficient.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
79
Evaluation Directorate – SPPB, July 2010
Finding 12: The proxy indicator used by CIDA excludes some program-related costs, which may under-
state the real costs of program management.
Finding 13: Despite the CPDS‟ stated emphasis on sectors and themes, most of the Malawi Program‟s
resources have been allocated to project needs. This has limited the potential for strategic
management, synergy, and learning within the program.
Finding 14: Until 2004, CIDA managers fostered cooperation and coordination among all stakeholders
involved in the Malawi Program to build the program. Since 2006, a significant amount of
staff time has been used to manage existing investments and generate requested information
to inform the program‟s future.
Finding 15: Canada‟s initial investment in establishing a PSU greatly facilitated the start up of its aid
program in Malawi in 1998. The relative and potential added value of this investment has
declined since 2006.
Finding 16: After 2004, DFAIT provided some administrative support services to CIDA in Malawi, but
the lack of a formal and well understood agreement caused tensions and over time
contributed to some inefficiency in the program.
Finding 17: CIDA‟s development approach in Malawi has been generally congruent with the principles
of aid effectiveness outlined in the Paris Declaration at the project level but less so at the
program level.
Finding 18: At the project level, CIDA management paid most attention to gender equality; other cross-
cutting issues received mixed levels of attention.
Finding 19: The CIDA Malawi Program did not establish specific objectives or strategies or allocate
resources and responsibilities to systematically address cross-cutting issues at the program
level. This makes it difficult to track and assess the program‟s overall ffectiveness in
mainstreaming these issues.
Finding 20: To launch a country program in Malawi in 1997-1998, CIDA crafted a focused, distinct,
innovative, and bold development strategy that was carefully designed to maximize the
impact of Canada‟s relatively modest resources.
Finding 21: The 1998 CPDS guided CIDA‟s bilateral investment decisions in Malawi up to 2004. Since
2004, it has had less influence over the program‟s investment decisions and directions,
particularly since there were limited resources for new programming.
Finding 22: The Malawi Country Program LFA has had limited utility as a basis for capturing and
measuring the program‟s expected performance beyond the project level.
Finding 23: The CIDA Malawi Program‟s somewhat atypical management structure allowed the
program to become operational quickly. However, the approach was not sustainable and
created some lasting management difficulties and risks.
Finding 24: A vacuum in CIDA leadership in 2004/05 negatively affected the overall performance of
CIDA‟s Malawi Program. This situation may be an anomaly, or it may be indicative of
problems faced by other small and less-visible CIDA country programs.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
80
Evaluation Directorate – SPPB, July 2010
Finding 25: CIDA‟s RBM and risk management principles have been widely respected at the project
level, but less so at the program and sector levels.
Finding 26: The Malawi government‟s limited experience with program-based approaches, combined
with CIDA„s limited financial resources for new programming after 2006 warranted CIDA‟s
use of a mix of program implementation and delivery mechanisms in Malawi.
The investment in the HIV/AIDS SWAp was the lone PBA in Malawi. Although donors and developing
countries are paying increased attention to program-based approaches, which are in keeping
with the principles of the Paris Declaration, the evaluation found that the Malawi program‟s
mix of approaches was very appropriate in the Malawi context during the period.
Recommendation 1: The managers of the CIDA Malawi Program should identify efficient ways to deliver
this modest presence country program.
C I D A M a l a w i C P E S y n t h e s i s R e p o r t
81
Evaluation Directorate – SPPB, July 2010
AA pp pp ee nn dd ii xx XX II CC II DD AA '' ss MM aa nn aa gg ee mm ee nn tt RR ee ss pp oo nn ss ee
CIDA Malawi Program Recommendation
Commitments and Action Responsibility Centre
Planned Completion
Date
Progress
1. The managers of the CIDA Malawi Program should identify efficient ways to deliver this modest presence country program.
Agreed. The Malawi Program has reduced the size of its Program Support Unit and is currently examining its structure at CIDA headquarters and in the field to ensure that the program is cost effective and efficient. CIDA will continue to carefully select its development partners in line with the Government of Canada contractual regulations, seeking those who can represent CIDA‟s development interests in Malawi, while also effectively managing programming.
RDG, Southern and Eastern Africa
Directorate
31 Dec. 2010 Projects are currently under development. Alternative management structures will be explored during 2010.