ForewordThe evaluation of public financial management reform is one of several joint evalua-tions, undertaken under the umbrella of the OECD’s Development Assistance Com-mittee, which are focused on issues identified as key for using country systems andwhere looking at donor assistance collectively makes more sense than trying to attrib-ute results to a single actor.The evaluation involved three main components, a literature review (publishedin 2009) a quantitative study (published in 2011) and finally three country casestudies. It is through the three cases – Burkina Faso, Ghana and Malawi – that theevaluation has been able to look in detail at the context and mechanisms that makePFM reforms successful.The importance of good public financial management for the effectiveness ofthe state has become increasingly clear over the years. Good public financial man-agement supports not only good governance and transparency but is also crucial foreffectively delivering the services on which human and economic development rely.For these reasons, many bilateral organisations and multilateral institutions consid-er public financial management to be a priority. The evaluation units of the AfricanDevelopment Bank (AfDB), the Swedish International Development Agency (Sida)and Danish International development Assistance (DANIDA) commissioned thisevaluation on behalf of a larger group of donors.We are now entering a second generation of PFM reforms, so learning the les-sons from past experience is crucial. The added value of the country case studies isthat they analyse the context and mechanisms which make for successful PFMreforms. The report identifies lessons for countries going through PFM reforms,including the importance of high-level buy-in and leadership, alongside effectivecoordination. It also identifies lessons for development partners, which includeresisting the temptation to push for reforms where the context is not right, and mak-ing sure the advice they provide is high quality and relevant to the setting. Theevaluation observes a general improvement in donor coordination and alignment,while noting that it is those inputs which are not integrated into government-ownedprogrammes that most often fail. And on both sides, flexibility is needed – even thebest planned projects often need adjustments.Beyond this evaluation, the challenge now – for both countries going throughreforms and for their development partners – is to apply those lessons in practice, toget better results in future.The three organisations that commissioned the evaluation and the teams thatcarried it out would like to express sincere thanks to those individuals and groupsthat played a role in the evaluation process. In particular, this third phase of theevaluation would not have been possible without the cooperation of governmentofficials and local PFM experts in the three country case study countries.5

Acronyms and AbbreviationsAAPAFDAfDBAfroSAIBCEAOBPEMSCABRICAGDCAPA/FPCdCCDMTCFCFAACIDCIDACIECIRCOCPARCPIACRSCSLPCSOsDACDanidaDFIDDGB(HIPC) Assessment & Action Plan (for PFM)Agence Fran§aise de DéveloppementAfrican Development BankAfrican Organisation of Supreme Audit InstitutionsBanque Centrale des Etats d’Afrique de l’OuestBudget and Public Expenditure Management SystemCollaborative African Budget Reform InitiativeController and Accountant General’s DepartmentCadre Partenarial d’Appui au renforcement des Fi-nances PubliquesCour des Comptes (Court of Accounts: the SupremeAudit Institution)Cadre de dépenses à Moyen Terme (MTEF)Contrôleur FinancierCountry Financial Accountability AssessmentCircuit Intégré de la DépenseCanadian International Development AgencyComptabilité Intégrée de l’EtatCircuit Intégré des RecettesControlling OfficerCountry Procurement Assessment ReviewCountry Policy & Institutional AssessmentCreditor Reporting System (of the OECD-DAC)Cadre Stratégique de Lutte contre la Pauvreté (PovertyReduction Strategy)Civil Society OrganisationsDevelopment Assistance Committee (of the OECD)Danish International Development AssistanceDepartment for International Development of the UKDirection Générale du Budget

Direction Générale des ImpôtsDirection Générale du Trésor et de la ComptabilitéPubliqueDevelopment PartnerDevelopment Policy LendingDirection des Services InformatiquesEcole nationale des régies financières(National Financial School, Burkina Faso)European UnionFinancial Administration ActFinancial Administration RegulationFrancs de la communauté financière africaineGeneral Budget SupportGovernment of Burkina FasoGross Domestic ProductGovernment Finance StatisticsGovernment of GhanaGovernment of MalawiGhana Integrated Financial Management InformationSystemGhana Joint Assistance StrategyGross National IncomeGrowth and Poverty Reduction StrategyGerman Technical CooperationHuman Development IndexHighly Indebted Poor CountriesInternal Audit AgencyIndependent Evaluation Group (World Bank)Integrated Financial Management SystemIdentifiant Financier Unique (Tax Identification Number)International Monetary FundInternational Organisation of Supreme AuditInstitutionsIntegrated Personnel and Payroll Database systemInternational Public Sector Accounting StandardsLocal Government Authority7

Ministries, Departments and AgenciesMulti-Donor Budget SupportMillennium Development GoalMinistère de l’Economie et des FinancesMinistère des Finances et du BudgetMinistry of FinanceMinistry for Financial and Economic PlanningMalawi Growth and Development StrategyMembers of ParliamentMedium Term Expenditure FrameworkNational Audit OfficeNational Democratic CongressNational Development Planning CommissionNon-Governmental OrganisationNew Public ManagementNew Patriotic PartyOpen Budget InitiativeOfficial Development AssistanceOffice of the Director of Public Procurement (Malawi)Organisation for Economic Cooperation & DevelopmentPublic Accounts CommitteePerformance Assessment FrameworkPublic Expenditure & Financial AccountabilityPublic Finance ManagementPublic Institutional Development ProjectProject Implementation UnitPublic Procurement BoardPurchasing Power ParityPlan de Renforcement de la Gestion Budgétaire(Burkina Faso Plan of Action to Strengthen BudgetManagement)Poverty Reduction and Growth FacilityPoverty Reduction Strategy CreditPoverty Reduction Strategy (Paper)Public Service Commission

Table of ContentsExecutive Summary............................................................................... 121objectives and Approach.....................................................................181.1 Objectives of the evaluation .........................................................181.2 The research context for the evaluation.....................................291.3 The Tunis Inception Workshop ....................................................211.4 The Choice of Case Study Countries ...........................................221.5 Evaluation Framework ................................................................241.6 The Evaluation Questions & the Approach to the CountryStudies ...........................................................................................281.7 The Approach to the Synthesis process .....................................311.8 Limitations of the Analysis...........................................................321.9 Report Structure ..........................................................................34overview of PFm reform experience in the Study countries.........352.1. Economic and political background............................................352.2. Reform Inputs ............................................................................... 412.3. Reform Outcomes ........................................................................432.4. The PFM reform processes in overview: the resultingC-M-O combinations ....................................................................49Successful PFm reform: what is the right context?.........................603.1 Introduction...................................................................................603.2 Refining our Programme Theory................................................653.3 Why and how is political commitment to PFM reformimportant?.....................................................................................683.4 Understanding “Policy Space” constraints................................723.5 Additional contextual factors for further investigation.............75Successful PFm reform: what are the right mechanisms?............794.1 Introduction...................................................................................794.2 The most important aspects of the managementmechanisms for PFM reform ......................................................794.3 The continuing weaknesses in the coordinationof technical assistance.................................................................824.4 The lack of an effective learning and adaptation process ........85Conclusions and emerging lessons....................................................875.1 Key Conclusions ...........................................................................875.2 Lessons for Development Agencies supportingPFM reforms.................................................................................885.3 Lessons for Governments managing PFM Reforms ................90

table of Figures & tablesFigure 1.The Evaluation Framework and the place of theEvaluation Questions within the Intervention Logic ..................25Figure 2.Overview of the Case History Approach to the CountryStudies ...........................................................................................30Figure 3.Overview of PFM Performance in Study Countries,2001–2010......................................................................................43Figure 4.Burkina Faso PEFA Scores 2007 & 2010, comparedto median score for 100 countries ..............................................46Figure 5.Malawi PEFA Scores 2006 and 2011, comparedto median score for 100 countries ..............................................46Figure 6.Ghana PEFA Scores 2006 and 2010, comparedto median score for 100 countries ..............................................46Table 1.Preliminary Estimation of Relative Impact of DonorSupport to PFM reforms in SSA ..................................................23Table 2.The Selection of 9 Case Histories from the StudyCountries .......................................................................................31Table 3.Key Economic, Social & Political Indicators forthe Study Countries (2010) ...........................................................35Table 4.Ministers, Deputy Ministers and Members of Parliamentrelative to the population in Case Study Countries ................... 41Table 5.Funding of PFM Reform Inputs in the Case StudyCountries 2001–2010 ....................................................................42Table 6.Average PEFA Scores by Cluster for the Case StudyCountries, 2006/07 and 2010/11..................................................45Table 7.Overview of PFM Reform in the Case Study Countries:the C-M-O Combinations ............................................................. 61Table 8.Results of 2008/ 09 Afrobarometer survey questionregarding citizens’ attitudes to their Government.....................78Table 9.Burkina Faso: Ministers of Finance 1996–2011 .....................96Table 10.Malawi – Ministers of Finance 1994–2011 ............................96Table 11.Ghana: Ministers of Finance 1982–2011 ................................97

Executive SummaryThis Synthesis Report provides a summary of the conclusions of the JointEvaluation of Public Financial Management Reform, managed by theAfrican Development Bank, Denmark and Sweden. The report synthe-sises the results of Country Studies prepared for Burkina Faso, Ghanaand Malawi, based upon desk research and fieldwork, and followinga standardised evaluation framework. The selection of the case studiesand the design of the framework drew upon a previous literature review,an Approach Paper and an extensive quantitative study, examining thelessons that might be drawn for the design of PFM reform processes fromthe data available on 100 countries that have completed PEFA assess-ments.The evaluation looked at two main questions: (i) where and why do PFMreforms deliver results and (ii) where and how does donor support to PFMreform efforts contribute most effectively to results? Our conclusions andthe corresponding lessons for Governments and Development Partnersare detailed below.The evaluation framework and its origins are fully explained in Chapter1 of the report, while details of the results of the country studies for Burki-na Faso, Ghana and Malawi are presented in Chapter 2. Chapter 3 anal-yses the key features of the context for PFM reform, which emerge as crit-ical to its success, while Chapter 4 examines the critical features of themechanisms for the management and coordination of PFM reform.Chapter 5 presents the conclusions and the corresponding policy lessonsfor Governments and Development Partners.Where and why do PFM reforms deliver results?I.PFM reforms deliver results when three conditions coincide: there is a strong political commitment to their implemen-whentation, reform designs and implementation models are well tai-whenlored to the institutional and capacity context; and strong coordination arrangements – led by governmentwhenofficials – are in place to monitor and guide reforms.

commitment at the technical level was not sufficient to generate polit-ical commitment.III. External donor pressure and domestic pressure from the Legislatureor Civil Society will generally contribute to preserving political com-mitment for reform, where it already exists but, in the case studies, itproved insufficient to generate political commitment for PFMreform, where it was lackingIV. PFM reform designs and implementation models will almost inevita-bly have flaws; hence a learning process is essential to permit the con-tinuous evolution and adaptation of reform designs and models.Where management and coordination mechanisms for PFM reformbuild in adequate provision for the regular, independent evaluationof performance, these learning processes are more likely to be effec-tive.V. Reform outcomes are generally more favourable where a wide rangeof policy options is available at the outset or where the mechanismsfor monitoring and coordination of reforms promote active lesson-learning and adaptation during the implementation process. By con-trast, the case study countries frequently found themselves facinga constraint in respect of the policy space for reforms, where themenu of available policy designs and models for PFM reform was notappropriate to the institutional and capacity context, and where thelearning and adaptation processes were rarely effective enough topromote quick changes to faulty design and implementation models.VI. Advocacy work by CSOs and activism by the Legislature are morelikely to be useful, when focused on a narrow objective, such as theimprovement of budget transparency. In the case study countries, theinfluence of the Legislature and Civil Society on PFM reform provedlimited, in part because of the limited expertise of these stakeholdersin this regard but more significantly because of the relative absenceof a culture of public accountability.Where and how does donor support to PFM reform effortscontribute most effectively to results?I.Donor funding for PFM reform has facilitated its implementation inthose countries where the context and mechanisms were right forsuccess, and where external funding was focused on the Govern-ment’s reform programme. On the other hand, governments in thecase study countries showed a willingness to fund PFM reformsdirectly and their ability to do so was significantly facilitated by theGeneral Budget Support inflows they were receiving. Hence, inmany cases, direct external funding for PFM reform may not be thedeciding factor.13

II. Donor pressure to develop comprehensive PFM reform plans and toestablish clearly defined monitoring frameworks has been a positiveinfluence in countries receiving Budget Support.III. By contrast, attempts to overtly influence either the pace or the con-tent of PFM reforms through Budget Support conditionality havebeen ineffective and often counter-productive.IV. Donor promises to enhance the utilisation of country systems havenot generally advanced very far. In the case study countries, the latedisbursement of Budget Support and the partial use of country pro-cedures have been inimical to good public finance management.V. External technical assistance and advisory support helped toadvance the PFM reform processes in the study countries when theywere focused on clear objectives and outputs, directly linked to theGovernment’s reform programme. However, too many TA activitiesdid not fulfil these conditions: there is a need for all TA activities insupport of PFM reform to be explicit about their objectives and theiranticipated outputs and outcomes and to be subjected to independentevaluation on a more systematic basis.VI. The provision of poor advice and the promotion of inappropriatereform models by external agencies remain an unfortunate feature ofmany PFM reform programmes. Greater attention to the appropri-ateness of reform models is needed, within an adaptive, learningapproach to PFM reform implementation.Key lessons for Development Partners Be more discriminating in the provision of financial sup- port to PFM reforms.PFM reforms deliver results when there isa strong political commitment to their implementation, when reformmodels are tailored to the institutional and capacity context and whenstrong coordination arrangements – led by government officials – arein place to monitor and guide reforms. Where these conditions are notin place, PFM reforms are unlikely to be successful. In such circum-stances, external support would be more appropriately used to developcore PFM skills, and to undertake diagnostic work, which might raiseawareness at the political level of the need for reform. A lign support as closely as possible to the Governmentprogramme and avoid pursuing independent technicalassistance initiatives.In the country cases, externally financedsupport to PFM reform was most efficient and effective, when it directlyfinanced, or supported through technical assistance, actions and inter-ventions identified within the Government PFM reform programme.The least efficient interventions were those, which supported actionsoutside of the programme or only tangentially related to it. Thus,technical assistance and institutional support should focus on specific14

outputs to which there is a shared commitment, and should be com-bined with Budget Support, where appropriate. Ensure that aid policy and practise works in favour of thePFM system and not against it.Aid dependent countries facethe perpetual problem of having to adapt their domestic PFM systemsto the requirements of their external aid partners. In the study coun-tries – and elsewhere – significant problems have been created by aidmechanisms making partial use of government systems. Three par-ticular problems arose, which undermined the good management ofpublic finances in the study countries: (i) the late disbursement of budgetsupport; (ii) the imposition of special reporting requirements for “basketfunds” or “trust funds” managed through the national budget process;(iii) the opening of special project accounts outside of the Single Treas-ury Account. Ensure that advice is up to date and informed by the ex-perience within country, within the region and by widerinternational experience.External support can play a useful role inbringing to bear new and more widely informed perspectives on PFMproblems, with which the Government is struggling. By opening “policyspace” in this way, it can help to resolve problems but when externaladvice is not well informed, it serves to close policy space. Externalagencies have a duty to ensure their advice is right, and, where this isnot immediately possible, to ensure that they work jointly with Govern-ment to learn from initial mistakes until an adequate solution is found. Ensure that internal procedures for the supervision andpeer review of initiatives to support PFM reform are effec-tive in providing acontinuous check on progress.Each of thecase study countries suffered from the continued implementation overseveral years of inappropriate reform models and approaches. Policyadvice will not always be right from the outset, in particular when work-ing on PFM reform issues where a degree of experimentation is oftennecessary, but it is important to ensure there are mechanisms in placeto ensure mistakes do not go uncorrected for too long. This requires thecreation – both within the Development Agencies and within Govern-ments – of a learning and adaptation culture, supported by a process ofcontinuous evaluation. Provide support, where necessary, to regional institutionsand professional associations working on PFM reform is-sues.In the case study countries, both regional governmental institu-tions – such as WAEMU – and regional professional associations – suchas CABRI and AfroSAI – were found to be influential in generatingimproved practises on public finance management. In so far as thescope of influence of such bodies could be expanded by more substan-tial external support, then clearly such investments would be of benefit.However, it should be recalled that much of the value of these bodiesderives from their ability to promote peer-to-peer learning: an excessive15

amount of external funding by DPs might undermine the effectivenessof this role. Continue to provide support to CSOs and Legislative bodieson PFM reform issues but accept that their inf luence mayonly be effective in the longer term.The experience of the casestudy countries suggests that CSOs and the Legislature are unlikely tohave significant effects on the pace and content of PFM reforms in theshort to medium term. However, broader international experience – in-cluding in the OECD countries – suggests that their influence over thelonger term may be important. Hence, support to such activities shouldbe continued but not as a substitute to direct support to the Executive.In addition, support should be concentrated on a narrower set of objec-tives, such as the improvement of public access to fiscal information.

Lessons for Developing Country Governments: It is essential to ensure clear and coherent support for PFM reform within the Executive and, over time to broadensupport across the political spectrum.PFM reforms are oftenperceived as purely “technical” measures and this perception needs tobe corrected. This must start within the Executive, with the Ministerof Finance and his/ her team working closely with the President and/or Prime Minister to promote reforms and then widening the scope ofconsultations to include the Cabinet and other members of the rulingparty. In time, it should be an objective to sensitise opposition membersto the need for PFM reforms, so as to ensure continuity over time, in theevent of changes of government. Serious attention needs to be given to the design and staff-ing of the structures established to coordinate and managePFM reforms.Those responsible for coordinating reforms shouldhave both technical competence and authority. The model of a techni-cal secretariat reporting directly to the Minister of Finance is a goodone. Another key feature of an effective model is that authority for im-plementation should be retained at the level of the relevant competentauthority (the President of the Court of Audit, the Directors General ofTreasury, the Budget, etc.) This will avoid doubts over the responsibilityfor implementation and will ensure that the coordinating body is notover-burdened with both implementation and coordinating/ monitor-ing responsibilities. Those responsible for coordinating PFM reforms should ex-ert control over external support to PFM and over dialoguewith Budget Support donors,related to PFM reform. This can bepromoted through the unification of responsibilities for attracting andmanaging external support to PFM reform with those for coordinatingimplementation by the departments and institutions of Government.

 structures established for monitoring PFM reformTheshould also evaluate performance in order to promotelearning from experience and the corresponding adaptionof implementation plans.PFM reform is inevitably complex andinitial plans are likely to need adaptation and adjustment. If implemen-tation of reform is to be efficient, the monitoring process must identifyreform bottlenecks quickly and take speedy corrective measures. Inorder to ensure this happens effectively, management structures mustembody not only monitoring of progress but also periodic – ideallyindependent evaluation of performance. Finally, the regular training of PFM staff needs to beaconsistent priority.The most important aspect of this is to ensurea consistent output of people with core skills in auditing, accounting,economics, procurement and financial management. In many coun-tries, investment needs to be made to re-establish PFM training institu-tions of adequate quality, and to ensure their recurrent funding overtime.

1. Objectives and ApproachThis Synthesis Report is submitted by Fiscus Limited, UK in collaborationwith Mokoro Ltd, Oxford. It summarises the key conclusions and lessons ofthe 2011 Joint Evaluation of Public Financial Management Reform, managedby the African Development Bank, Denmark and Sweden. The report synthe-sises the results of Country Studies for Burkina Faso, Ghana and Malawi1,based upon both desk research and fieldwork. It also draws upon an earlier lit-erature review2, an Approach Paper3and a Quantitative Study4, which exam-ined PEFA data and HIPC AAP data for 100 countries.The evaluation management group and the external peer reviewerreviewed the first draft of the Synthesis Report over October and November2011. Its results were also presented and discussed during January 2012 ineach of the case study countries. This final version of the Synthesis Report hasbeen revised so as to incorporate the comments received from each of thesesources. It is now expected that it will be the subject of a wider disseminationand consultation process, and it is hoped that it may also lead to further coun-try studies – in Africa, Asia, Latin America and Eastern Europe, whichshould serve to deepen its findings and clarify the extent of their applicabilityinternationally.

It has thus been a dual evaluation, involving both an evaluation of the overallprogrammes of PFM reform conducted over 2001 to 2010 in Burkina Faso,Ghana and Malawi and an evaluation of the external support provided tothese reforms by Development Agencies. This Synthesis Report focuses in1234Lawson, Chiche & Ouedraogo (2011), Betley, Bird & Ghartey (2011) and Folscher,Mkandawire & Faragher (2011).Pretorius, C. and Pretorius, N (December, 2008), Review of the Public FinancialManagement Literature, DFID, London.Lawson, A. and De Renzio, P, Approach & Methodology for the Evaluation of DonorSupport to PFM Reform in Developing Countries: Part A ( July 2009) and PartB (September 2009), Danida, Copenhagen.De Renzio, P., M.Andrews and Z.Mills (November 2010), Evaluation of DonorSupport to PFM Reforms in Developing Countries: Analytical Study of quantitativecross-country evidence, Overseas Development Institute, London.

1.2 ThE RESEARCh COnTExT FORThEEvAluATiOnThe evaluation forms part of a wider sequence of research activities, whichwere initiated by a management group comprising the evaluation depart-ments of the African Development Bank, DFID, Danida and Sida in order toaddress the gaps in knowledge regarding the design of effective PFM reformsand of effective external support to PFM reforms. As noted above, the initialresearch activities comprised a literature review, background analytical workto define an approach to the evaluation, and a quantitative analysis.The Literature Review(Pretorius & Pretorius, 2008) demonstrated theabsence of cross-country evaluation and research work to assess the effective-ness of PFM reforms. It identified and reviewed a substantial number of singlecountry assessments of PFM reform efforts, as well as a smaller number ofcross-country analyses of specific types of reforms – such as Medium TermExpenditure Frameworks (MTEFs), but found no existing comprehensivecross-country evaluations. It concluded that, beyond broad generalisations –drawn essentially from evaluations of structural adjustment processes andbroader public sector reforms, there was little knowledge of what made PFMreform efforts more or less successful and of what made financial and techni-cal support to PFM reforms more or less effective.Given this important gap, theApproach Paper(Lawson & De Renzio,2009) focused on deepening the literature review initially conducted, onextending the debate through a structured process of consultations, and ondefining more carefully how success in PFM reform could be defined andmeasured. Drawing on the literature on budget institutions across differentdisciplines, it provided a definition of three key dimensions that could be usedto track the impact of budget reforms over time, namely transparency andcomprehensiveness, the quality of links between budgets, plans and policies,and the quality of control, oversight and accountability. It considered themain sources of data on budget institutions, and demonstrated that by com-bining the results of IMF/World Bank HIPC tracking studies with PEFAassessments, the changes in the quality of budget institutions could be meas-ured against these criteria for 19 developing countries over the period 2001–2007. This background analytical work thus paved the way for the quantita-tive study undertaken during 2010 and also laid out the elements of the evalu-ation framework, later developed in the Inception Report to this evaluation5.Theanalytical study of quantitative cross-country evidenceofthe impact of PFM reforms was completed in November 2010 (De Renzio,5Lawson, A (2011), Joint Evaluation of Public Finance Management Reforms inBurkina Faso, Ghana and Malawi: Inception Report, Fiscus Limited and Mokoro:Oxford, UK.

Andrews & Mills, 2010). It drew on information from PEFA assessmentsundertaken in 100 countries over 2006–2010, financial data on donor supportto PFM reforms collected from the donor agencies most active in this area,and a large data set on economic/social, political/institutional and aid-relatedvariables. Its key findings were as follows: conomic factors are most important in explaining differences in the qual-E ity of PFM systems. Specifically, countries with higher levels of per capitaincome, larger populations and a better recent economic growth record arecharacterised by better quality PFM systems. By contrast, state fragility,has a negative effect on the quality of PFM systems. onor PFM support is also positively associated with the quality of PFMD systems. On average, countries that received more PFM-related technicalassistance have better PFM systems. However, the association is weak. he share of total aid provided as general budget support is positively andT significantly associated with better PFM quality. Thus, the choice of aidmodalities contributes to explaining differences in the quality of PFM sys-tems in the poorer countries where donor efforts are concentrated. he level of donor PFM support is more strongly associated with improvedT scores forde jureand concentrated PFM processes, highlighting how donorPFM support seems to focus more on rules, procedures and specific actorswithin government. Results are reversed when it comes to upstream vs.downstream processes. Here, the association is stronger with downstreamprocesses, possibly highlighting the large amounts of funding devoted toIFMIS projects, a typical downstream PFM reform.The study emphasised that these results suffered from a number of limitations,including weaknesses in data quality and problems in interpreting causalityrather than merely association. The study authors accordingly stressed theneed to interpret the results with a lot of caution, and noted that the results‘highlight the need to complement these quantitative findings within-depth qualitative research at country level’.There is thus a direct link between the quantitative study and our countrycase studies in Burkina Faso, Ghana and Malawi. This is reflected in theselection of case studies, in the evaluation questions and in the overall evalua-tion framework, which is presented below. This linkage is emphasised in twoparticular lines of investigation: irstly, the quantitative study confirmed, in general terms, the observationsF made in earlier analysis of countries with PEFA assessments (Andrews,2010), that‘countries make budgets better than they execute them, pass laws better thenthey implement them and progress further with reforms for which responsibility lies witha concentrated group of actors in the Ministry of Finance.’The country studiesallowed a qualitative investigation of the factors explaining these trends. econdly, the most significant uncertainty thrown up by the quantitativeS study related to the direction of causality between high levels of PFM relat-ed donor investment (both in the form of Budget Support and as support toPFM reform) and PFM performance. The study found an association butwas unable to determine, whether it simply reflected the tendency of donor

1.3 ThE TuniS inCEPTiOn WORkShOPIn order to initiate activities under the evaluation, on the 3rdand 4thMay2011, PFM experts and practitioners from across Africa were invited by theAfrican Development Bank to Tunis to share PFM reform experiences.22 African government representatives from 10 separate countries6talkedopenly about their PFM reform experiences. The points below reflect the keyobservations made by participants7: uccessful PFM reform depends on political support and lead-S ership:reform works well when it has strong political leadership. Hence,it is essential to take into account the political context when designingreform. Coordination between the technical level and the political level iscrucial. Communication of PFM reform across government and transpar-ency over reform results and challenges can be beneficial in building sup-port. Reforms need to be ‘sold’ to the politicians (including the opposition)especially when the political situation is fluid and changing. eforms must be country owned:reform should not be imposed asR a donor condition. While it seems to be the case that reform happens fasterwhen linked to budget support, it is more likely to be sustained if ownership isdeveloped. Without ownership, reform stops when the money stops. lan big, implement small:The “Big bang” approach has advantagesP as everything related to the PFM cycle moves together, although it seems bestto design the whole picture but implement piece by piece. It is important toconsider the breadth, speed and depth of PFM reform. The notion of the “Bigbang” suggests all of these, whereas African experience suggests the need forbreadth – the need to be holistic, covering the whole PFM system, while im-plementing gradually based on priorities and human capacity, and choosinga ‘depth’ of reforms (degree of sophistication), which is consistent with the realneeds and the capacity to implement. ake account of existing capacity:PFM reform is capacity constrained.T It tends to work well when donors finance the initial stages of the PFM reformprocess but then stand back and allow reform to occur in line with existingcapacity. .. and of the magnitude of the change management involved:. Change management aspects of PFM reform are often neglected. Reformprogrammes tend to under-estimate the time and effort involved in fomentingsustainable changes in work culture and work practises.

Therefore, in relation to the direction of causality between high levels of PFMrelated donor investment (both as Budget Support and as support to PFMreform) and PFM performance, the African government officials who partici-pated in the Tunis conference shared two perceptions relevant to this ques-tion. Firstly, there was unanimous agreement that successful PFM reformrequired political support and leadership and needed to be country-owned.The question of whether donor investment in PFM reform could help to gen-erate political support was not explicitly discussed, however. Secondly, theview was expressed that the pace of reforms needed to be consistent withcapacity and that external support often encouraged a pace of reform thatexceeded domestic capacity. The case studies allowed each of these themes tobe explored further.

1.4 ThE ChOiCE OF CASE STudyCOunTRiESThe 2009 Approach Paper for the evaluation8recommended the use of a pur-posive methodology for the selection of the country case studies for the evalua-tion. Bringing together data from the 2001 and 2004 HIPC AAP assessmentsand the early PEFA assessments undertaken up to 2007, the Approach paperfirst classified the 14 African countries for which this data was available intotwo groups, comprising countries where budget institutions appeared toimprove over the period and countries where budget institutions appeared notto have improved or to have deteriorated. Secondly, drawing on data from theOECD-DAC CRS database, an estimation was made of the donor inputs toPFM reform provided per capita to these same countries over 1998–2007. Theresulting table is presented below.

This analysis suggested that there were only three countries (Burkina Faso,Tanzania and Zambia) where donor support had been associated with suc-cessful reforms and three others (Ethiopia, Ghana9and Mali) where reformsappeared to produce positive outcomes even in the presence of more limiteddonor effort. The Approach Paper stressed that no firm conclusions could bedrawn from ‘this simple comparative exercise’ but it suggested that it pointedto ‘a range of situations which merited further investigation in the selection ofcase studies’.Specifically, the Approach paper suggested that, ‘excluding the countriesin the shaded box, the ideal way of applying the evaluation framework wouldbe to select a limited number of case studies focusing on countries from theremaining three boxes, possibly pairing them in ways that keep constant fac-tors such as the administrative heritage (for example, by pairing Francophoneand Anglophone countries). In this way, it would be possible to examine threetypes of situations:a) One in which donor support appeared to be positively correlated with PFMimprovements;b) One in which donor support appeared to be negatively correlated with PFMimprovement;c) One in which significant PFM improvements appear to have occurred de-spite relatively low levels of donor support.’

The Evaluation Department of the African Development Bank thenapproached these 14 countries to enquire which would be interested in collab-orating with the evaluation process, in order to find ways of strengtheningtheir PFM reform programmes and of improving the quality of external sup-9 Various PFM specialists consulted on the Approach Paper expressed surpriseat the resulting categorisation of Ghana, suggesting that it was misleading intwo respects. Firstly, there appeared to be a significant under-recording ofthe level of donor funding for PFM reforms. Secondly, the results of the 2001HIPC AAP were considered by many to have been unfeasibly low, giving animpression of significant improvements in subsequent years, which was notconsistent with other reports on the status of PFM systems in Ghana.

port to those programmes. Three countries agreed to collaborate in the pro-cess, which appeared at first sight to fall into these three groups, namelyBurkina Faso [Group a)], Malawi [Group b)], and Ghana [Group c)].Information from the case studies themselves has demonstrated that someof the preliminary estimates underlying Table 1 were in fact wrong. In partic-ular, all three countries were found to have been recipients of relatively highlevels of donor support to PFM. In addition, more recent PEFA assessmentsundertaken in 2009, 2010 and 2011 convey a more nuanced picture of the per-formance of PFM systems in each of these countries. Nevertheless, the casestudies do present interesting variations in the performance of PFM reformsas well as some striking similarities. A fuller description is presented in Chap-ter 2.

1.5 EvAluATiOn FRAMEWORkThe evaluation framework was defined in the Inception Report for the evalua-tion (Lawson, 2011). It utilises the OECD-DAC evaluation criteria and thusassesses the relevance, efficiency, effectiveness and sustainability both of coun-try programmes of PFM reform and of the external support provided to thoseprogrammes. It gives particular attention to three dimensions of PFM reformprocesses: heContexts,in which reforms have taken place;T  heMechanismsadopted for the design, management and delivery ofT reforms; and he consequentOutcomesachieved.T Success is associated with improvements in the quality of budget systems, asmeasured primarily by changes in the Public Expenditure and FinancialAccountability (PEFA) assessment framework indicators and the narrativePEFA reports. The evaluation framework characterises these changesasintermediate outcomesin a ‘PFM Theory of Change Framework’, whichis presented in Figure 1 below10. The Framework details PFM Reform inputs,outputs and intermediate outcomes and examines the relationship betweenthem, and with key contextual factors.

Inputsare defined as the resources and other inputs provided in order to pro-mote PFM reform. These are divided between direct funding by governmentsto internal PFM reform efforts, external funding by Development Partners(DPs) of PFM reform efforts and complementary inputs by DPs. These com-plementary inputs might be aimed at facilitating better PFM through the useof country systems and the provision of budget support, or at improving thedesign and implementation of PFM reforms through policy dialogue andexternal monitoring (often linked to budget support).Fourcontextual factorsare considered. One is internal and relates topublic and civil society pressure for PFM reform11. It examines the relativeimportance for the design and implementation of PFM reform of the votingpreferences of the electorate (to what extent good PFM is prioritised by voters),of research and advocacy work by CSOs, of political lobbying by CSOs andothers, and of regional and international norms for PFM, which might pro-vide “rallying points” for civil society groups and for the design of advocacycampaigns12.Also included within the contextual factors are external constraints – con-ceptualised as political, financial and policy space constraints. These externalconstraints are seen to impinge on the PFM reform ‘production function’, inother words on the capacity of PFM reform inputs to generate the plannedoutputs. Political constraints relate to the degree of political ownership andsupport for PFM reforms. Financial constraints relate to the ability to financePFM reforms in the face of competing priorities, and “Policy space con-straints” relate to the nature of policy ideas, which might potentially be con-sidered in designing PFM reforms. This latter constraint reflects the prevail-ing conventional wisdom over the types of reforms that could be contemplatedand the influence on that conventional wisdom of government stakeholdersand academic and civil society organisations, as well donor agencies andinternational and regional institutions. Another potential way of describingthis would be in terms of the “space for novelty”13in PFM reform policies andstrategies.Outputsare defined as the immediate changes in the architecture andsubstance of the PFM system generated by the combined set of inputs. Theseare categorised into four groups: i) Changes in human resource endowments(people and skills); ii) Changes in laws, procedures and rules; iii) Changes in11 Advocacy work by CSOs in support of greater transparency and accountability mayattract funding support from Development agencies and international foundations. Inthis sense, it is an “output” of PFM reform efforts. However, such funding is modestin relation to the direct funding available for core components of the PFM reformprogramme. Hence, in the case studies, funding to CSOs for advocacy work was notincluded within the financial estimates of PFM reform inputs and it was dealt withpurely as a contextual input.12 For example, amongst the country case studies, the West African Economic & Mon-etary Union (WAEMU or UEMOA in French) establishes both convergence criteriafor macroeconomic and fiscal management, and norms for certain aspects of PFM, towhich its eight members subscribe, including Burkina Faso.13 This is considered in Pritchett, Woolcock & Andrews (2010) as a key characteristic bywhich to judge systems.

The framework uses the PEFA assessment framework to measure changes ineach of these clusters of PFM functions, based on a categorisation of the sub-dimensions of the PEFA indicators between each of these clusters. The catego-risation14is based on Andrews (2010) and was also applied by De Renzio et al(2010) in the quantitative study, which examined data on 100 separate coun-tries which had completed PEFA assessments between 2006 and 2010.The PEFA Performance Measurement Framework for PFM (PEFA 2005)is to date the most comprehensive attempt at constructing a framework toassess the quality of budget systems and institutions. It comprises 28 indica-tors, which assess system performance at all stages of the budget cycle, as wellas crosscutting dimensions and indicators of budget credibility. It also includesthree additional indicators on donor practices. (See www.pefa.org.)In order to derive the PEFA scores by budget “cluster”, Andrews (2010)and De Renzio et al (2010) follow four steps, which were also used in the coun-try studies to convert scores from the PEFA assessments in each country intonumerical averages for each cluster. De Renzio et al (2010, pp. 11–12) explainthe methodology very clearly:“First, we only considered indicators PI-5 to PI-28, as indicators PI-1 to PI-4 coverPFM system outcomes and performance, and not the quality of PFM systems per se. Second,for multi-dimensional indicators we used sub-indicator/dimension scores rather than sum-mary indicator scores in order to fully exploit the information contained in the PEFA scores.This also allowed us to avoid the downward bias introduced by the M1 scoring methodol-og y, where summary indicators are based on the lowest scoring dimension, or ‘weakest link’.Third, we converted the letter scores included in PEFA reports into numerical scores, withhigher scores denoting better performance (from A=4 to D=1). Fourth, we constructed ourdependent variable in three different ways: 1) as an overall simple average of the 64 numeri-cal scores that include all sub-indicators/dimensions for indicators PI-5 to PI-28; 2) asaverages of numerical scores for sub-indicators/dimensions in each of six clusters of indica-tors grouped by phase of the budget cycle. This generates six sub-indices to be used separatelyas dependent variables; 3) as individual scores for each of the 64 sub-indicators/dimensionsin indicators PI-5 to PI-28. This generates a panel-type dataset of64 dimensions * 100 countries.”14 These clusters are slightly different from the ones included in the PEFA methodology,as they have been rearranged to increase their level of internal consistency. For furtherdetails, see Annex 3, as well Andrews (2010:8) and Andrews (2007).

1.6 ThE EvAluATiOn QuESTiOnS &ThEAPPROACh TO ThE COunTRy STudiESEach of the Country studies was based on twelve evaluation questions, whichare presented in Box 1 below15. The evaluation questions were structured so asto provide a standardised framework for assembling evidence, so that theresults of the country studies could be easily synthesised to provide answers tothe overall high-level questions.Each of the Country Studies generated a set of“case histories”ofchange in PFM systems. The aim of this approach was to ensure that the eval-uation did not miss patterns of change, which might be obscured by looking atthe average changes in the system as a whole. For example, even in a countrymaking limited progress with its overall PFM reforms, there would probablybe specific sub-components progressing faster. This is unlikely to be a merecoincidence: it would be more likely to reflect the relative balance for thosesub-components of the positive and negative forces, driving or blockingchange. By examining the patterns of change within the sub-components ofthe PFM reform programme, we believe that more significant insights havebeen revealed. The approach is illustrated in Figure 2 below.

Box I. The Evaluation questionsA. Inputs & context: the design of PFM reformEQ 1:What has been the nature and the scale of PFM reform inputs provid-ed by Government and by Donors?EQ 2:What types of structures have been used for the design and manage-ment of these reform inputs? Have these structures served to provide a coordi-nated and harmonised delivery framework?EQ 3:What types of complementary actions have Donors taken to supportPFM reforms and what has been their significance? Have they had any influ-ence on the external constraints to reform?EQ 4:To what extent has there been domestic public pressure or regionalinstitutional pressure in support of PFM reform and what has been the influenceon the external constraints to reform?EQ 5:How relevant was the PFM reform programme to the needs and theinstitutional context? Was donor support consistent with national priorities? Towhat extent were adaptations made in response to the context and the changingnational priorities?

1.7 ThE APPROACh TO ThE SynThESiSPROCESSThis Synthesis report has thus brought together information from 3 countryhistories and from 9 case histories at the “PFM cluster” level. The objectivehas been to address the high level questions identified in the terms of refer-ence:‘The purpose of the evaluation is to identify what factors–institutional and contextual– contribute to successful PFM reformand how donors can best support PFM reform given the influenceof contextual factors on the process of change.’(Terms of Reference,pp.3–4)The adopted approach to analysis is that of ‘realist synthesis’ (Pawson &Tilley, 1997; Pawson 2002). The starting assumption of the realist school isthat most programmes of reform or social change work only in limited cir-cumstances. Therefore, the discovery and documentation of the ‘scope condi-tions’ within which a programme works becomes the main objective of theprocess of synthesis. Realist synthesis does this through the analysis of change

Mechanisms (M) working within different Contexts (C) and producinga range of Outcomes (O). By careful examination of these C-M-O combina-tions, it is possible to define to a certain level of detail the “boundary condi-tions” within which a programme theory will work, leading to a more tailoredtheory and to a better understanding of its transferability.Thus, the systematic analysis of a range of C-M-O combinations producesa continual refinement of programme theory, by learning both from success(favourable outcomes) and failure (unfavourable outcomes). In this sense, real-ist synthesis builds on the ideas of Karl Popper (Popper, 1959) regarding theimportance of being able to falsify a theory. Where a theory is open to falsifi-cation, it then becomes possible to refine the theory through the experience offailure. Learning from mistakes through ‘analytical induction’ (Lindesmith,1968) is essential to the process.The evaluation questions provide a template for documenting the C-M-Ocombinations generated by the case histories: ontextis captured by the questions relating to complementary DP inputsC (EQ3) and demand-side pressures (EQ4), as well as to binding externalconstraints (EQ8). echanismsare documented through questions on inputs (EQ1) andM outputs (EQ6) and on efficiency (EQ7) as well as through more detailedexamination of the structures for design and management of reform inputs(EQ2). utcomesare directly documented (EQ9) and also examined withO respect to their relevance for service delivery, especially for women andvulnerable groups (EQ10). elationships between Context-Mechanism-OutcomesareR examined from the perspective of relevance (EQ5), effectiveness (EQ11)and sustainability (EQ12.)By examining these different combinations, at the country level and then atthe PFM cluster level, we have been able to develop a tailored programmetheory regarding firstly the critical aspects of context, which contribute to suc-cessful PFM reform, and secondly the most important features of the mecha-nisms adopted for delivering PFM reform. This analysis has allowed us tocrystallise a number of lessons, which are presented in our conclusions.

covered has been further broadened. We have also drawn comparisons withthe quantitative study and with the existing literature on PFM reform wher-ever possible. Nevertheless, it is a small sample from which to develop a pro-gramme theory.It is therefore vital to disseminate and discuss the results of this work, inorder to assess whether there are other PFM reform experiences, which havenot been accessed and which might serve to enrich the conclusions. It is alsoimportant to assess whether the insights here presented are supported by theimpressions and by the experiences of seasoned PFM practitioners andresearchers. The dissemination process envisaged by the Evaluation Manage-ment Group should allow for the results to be tested through such feedback.Moreover, in the medium term, it is essential that this study should becomplemented by further case study work, which might build upon itsinsights. Wherever possible, additional case study work should deliberatelyseek to test the “boundary conditions” of the conclusions reached, by choosingcontrasting case study countries. If additional case study work were to utilisethe same evaluation framework and the same approach to the synthesis ofresults, this would permit the continuous updating of the “programme theo-ry” here presented.In terms of the approach to fieldwork, the results are also subject to a num-ber of limitations. Most notably, fieldwork comprised only 6 person weeks ofwork (three consultants for two weeks) and within such a short time period, therange of stakeholders who could be interviewed was, of course, limited. Asa consequence, the depth of analysis possible for the three detailed cases stud-ies undertaken in each country was uneven, although in our opinion adequateto justify the resulting conclusions. It is also the case that the analysis mighthave been biased by the perspectives of the stakeholders directly involved inthe reforms, as the government and donor staff managing and undertakingthe PFM reforms in each country comprised the majority of those inter-viewed. Whilst members of Civil Society Organisations dealing with PFMissues were interviewed in each country, either individually or within focusgroups, they were relatively limited in number. With regard to members of theLegislature, it did not prove possible to interview MPs in Burkina Faso andonly small numbers in Ghana and Malawi.On the other hand, in drawing conclusions from fieldwork, the evaluatorswere careful to triangulate findings wherever possible. For many issues, sys-tematic comparisons of the perspectives of central agency staff, sector minis-try staff and donor representatives were possible. In other cases, comparisonswere made between the views of different donors or the opinions of public sec-tor managers and junior staff. The fieldwork also built on detailed desk stud-ies17that were undertaken prior to the fieldwork, drawing on the extensivedocumentation available on each of these countries either on the Internet or17 The desk studies were undertaken by experienced research assistants, who wereclosely supported by the other team members for each country. Reports were allpublished in advance of the field work and amounted to documents in excess of 100pp each including annexes: Chiche, M., June 2011 (Burkina Faso), Faragher, R. June2011(Malawi) and Gordon, A. & Betley, M, June 2011 (Ghana).

2. Overview of PFM reformexperience in the Study countriesIn this chapter, we present an overview of our main findings, before analysingmore deeply in chapters 3 & 4 the contextual factors and the aspects of thePFM reform design and delivery mechanisms, which have proved critical. Inorder to set the scene, we begin first with a simple comparative analysis of theeconomic and political frameworks in each of the three countries. We thenconsider the relative levels of inputs to PFM reform provided in each coun-try – both directly by Governments and by Development Agencies – and theoutcomes achieved, before presenting a summary of the C-M-O combinations(Context-Mechanism-Outcomes) generated by the three country studies andby the 9 more detailed case histories.

World Bank Development Indicators 2010;Human Development Report 2010;OECD DAC Database, accessed 10/1/11;Freedom In the World 2010, Freedom House;Corruption Perceptions Index 2010, Transparency InternationalThe ratings process is based on a checklist of 10 political rights ques-tions. Scores are awarded to each of these questions from which a rating of1 to 7 is derived, with 1 representing the highest and 7 the lowest level offreedom.The ratings process is based on a checklist of 15 civil rights questions. Again,a rating of 1 to 7 is derived, with 1 representing the highest and 7 the lowestlevel of freedom.The CPI measures the degree to which public sector corruption is perceivedto exist within a country on a scale of 0 (highly corrupt) to 10 (very clean).

Economic and Social contextTable 3 above shows key economic, social and political indicators for each ofthe study countries. There are certain similarities but also important differ-ences. They are all high recipients of aid and each can be described in a broadsense as a Developing Country but income per capita is considerably higher inGhana – a coastal, natural resource rich country, which became formallya middle income country in 2011 - than it is in Burkina Faso or Malawi, bothland-locked low income countries, which are predominantly reliant on rain-fed agriculture.Their recent economic development paths have also been quite different inseveral important respects: urkina Fasohas been through a process of wide-ranging macroeco-B nomic and monetary reform over the last two decades. The change ofregime in 1987 signalled a move away from a socialist model to a moremarket-oriented economic policy. A devaluation of the currency in 1994resulted in a remarkable acceleration of growth, and in the last 20 yearsBurkina Faso has experienced higher economic growth and lower inflation

than the average in the WAEMU and Sub-Saharan African (SSA) coun-tries. Between 2000 and 2010, it maintained an average growth rate ofover 5.2 % per annum, and except for a short-lived spike following thedevaluation, average inflation has remained below 3 % since 1995. How-ever, with a population growth rate of 3.1 % per annum, and a continuingsusceptibility to droughts and floods and a vulnerability to regional con-frontation among its trading partners (such as in Cote d’Ivoire), this level ofGDP has not been sufficient to be transformational. Its economic perfor-mance has been steadily positive rather than exceptional18, although therecent growth in mining exports does open up new opportunities for thefuture19. hanaalso experienced major macroeconomic and structural reformG over the decade of 1985–1995, as the Rawlings government managed thetransition from a protectionist state-led model of development to a liberal-ised, export driven model and from military rule to civilian rule, culminat-ing in the 1992 Constitution and the 1996 elections. It was helped by nar-row but robust export growth, based upon gold and cocoa, and by thesteady growth of its services sector. The period since the mid ‘90s has beencharacterised by a “boom and bust” cycle, linked closely to the electoralcycle20. In 1999 and 2000, as the incumbent National Democratic Con-gress (NDC) Government reached the end of its term, public spending wasdramatically expanded, allowing the fiscal deficit to mushroom, financedthrough domestic borrowing, and inflation, which hit 41 % in 1999. Theincoming New Patriotic Party (NPP) government introduced a period offiscal consolidation and tight monetary policy, paving the way for growthin excess of 6 % in 2006 and 2007, and of 7.3 % in 2008. Yet, as the NPPgovernment reached the end of its second term, 2008 also saw big increasesin public expenditure, with negative effects on the fiscal deficit, whichreached 13.5 % of GDP, as well as on inflation, and domestic interest rates.The returning NDC government has introduced a new period of fiscalconsolidation over 2009 and 2010. Helped by the start of oil production in2010 and the related investment, growth rates of 6.6 % and 8.3 % are fore-cast for 2010 and 2011.

18 Steady economic growth, increasing social expenditures and improvements in ac-cess to basic services saw a decline in the incidence of poverty from 54 % in 1998 to46.4 % in 2003, and to an estimated 43 % in 2010. Sustained efforts and investmentshave resulted in positive trends in human development, with strong increases in grossprimary enrolment, use of health services, vaccination rates, percentage of assistedbirths, and an improved access to clean water but it remains one of the poorest coun-tries in the world, ranked 161$1-$2 out of 169 countries in the 2010 Human Develop-ment Index.19 In 2009, gold replaced cotton as Burkina Faso’s most valuable export. Mining isexpected to increase export earnings by 25 %, bringing $450 million in added fiscalrevenue between 2010 and 2015 (World Bank, 2010a.)20 In common with Burkina Faso, Ghana also averaged 5.2 % annual growth in GDPover 2000 to 2007 but annual fluctuations around this average were much greater,and inflation was considerably higher than in neighbouring Burkina Faso.

a land-locked country, with a narrow export base and a heavy dependenceon agriculture, which supports 80 % of the population – mainly via small-holder production. In common with Burkina Faso, it is also susceptible toweather shocks and suffered a severe drought and widespread food short-ages over 2001 and 2002. Unfortunately, in common with Ghana, its man-agement of fiscal and macroeconomic policies has been subject to severepolitical pressures, leading to a similar “boom and bust” pattern linked toits electoral cycle, but lacking the cushion of Ghana’s substantial naturalresources and more diversified economy, growth has fluctuated around anaverage of 4 % per annum, rather than 5.2 %. Following the end of one-party rule, Bakili Muluzu of the United Democratic Front (UDF) waselected as President in 1994, and successfully re-elected in 1999. However,his second term was characterised by macro-economic instability causedby unsustainable fiscal policies. A surge in public spending resulted inunsustainable increases in the budget deficit, high domestic borrowing andrising inflation. His appointed successor, economist Bingu Wa Mutharikawas elected in 2004. After the election however, President Mutharikarefused to appoint Muluzi-proposed ministers to Cabinet and presenteda clear stance on the need for governance and economic reforms. Thiscaused a breach with the party resulting in President Mutharika forminghis own party while in office, but facing a minority in parliament. Helaunched a crackdown on corruption and took steps to improve fiscal disci-pline, actions which restored the confidence and support of internationaldonors (Cooney, Wenderoth et al., 2010). After turning around Malawi’sfiscal, economic and food security crisis and five years of economic reforms,he was re-elected in 2009 and his party, the Democratic Progressive Party(DPP), won a strong parliamentary majority. However, there was a furthersurge in public spending leading up to the elections, with public spendingin 2008/09 reaching a record level of 38 % of GDP, generating a fiscal defi-cit after grants in excess of 5 % of GDP. Moreover, early signals suggest thatPresident Mutharika’s second term in office will not be as strongly associ-ated with political support for fiscal consolidation and PFM reform as thefirst21.

Political frameworkThe difference in the political contexts of the three countries is striking andgoes a long way to explain the differences in the quality of economic manage-ment. We will note below that it has also had a significant influence on thedegree of political commitment to the implementation of PFM reforms.Since the establishment of the new constitution of 1991,Burkina Fasohas been one of the most politically stable countries in Africa. President BlaiseCompaoré has headed the Government through consecutive elections in 1991,21 Most notably, Goodwill Gondwe, who had successfully spearheaded the fiscal consoli-dation process and the acceleration of PFM reforms during President Mutharika’s firstterm in office, was not reappointed as Minister of Finance.

1998, 2005 and 2010, and his party (Congress for Democracy & Progress –CDP) have won large majorities in the parliamentary elections of 2002 and2007. Although these elections have been contested22, the degree of oppositionboth during and between elections has been limited. This has allowed theGovernment to pursue a consistent set of economic, social and PFM reformpolicies, without much need for the bargaining and frequent compromise,which is a common characteristic of democratic government.Nevertheless the country is yet to experience political alternation throughthe electoral process and the lack of a credible political opposition remainsa concern. Rising social unrest in recent times, involving riots over food priceincreases, national strikes over salaries, student protests over poor pay andsporadic protests by the military and police forces have caused internal turbu-lence and led to Cabinet reshuffling in 2008 and early 2011.Notwithstanding the weaknesses in domestic accountability and the lim-ited nature of public political debate, Burkina Faso is perceived within theregion and internationally as a modernising, reformist country. The limiteddegree of political opposition in Burkina Faso has allowed for a high level ofconsistency in economic policy, and in the approach to institutional reform:notably, PFM reform has enjoyed a prominent role within the Government’sreform agenda since the mid 1990s. The limited degree of political oppositionhas also allowed the President to adopt a technocratic approach to Govern-ment, with both political and senior Government appointments being basedpredominantly on merit, rather than on the need to grant political favours.Both inMalawiand inGhana,the degree of competition in politics isfar higher. In 1994, Malawi saw a transfer in power from the Malawi Con-gress Party (MCP), which under Dr. Hastings Banda had held power virtuallysince Independence, to the United Democratic Front (UDF) of Bakili Muluzu,and in 2009 to the newly established Democratic Progressive Party of Binguwa Mutharika. In 2000, Ghana saw a transfer of power from the NationalDemocratic Congress party, at that time under President Rawlings to the NewPatriotic Party (NPP). Then, following close elections (including a run-offelection) in December 2008, a government led by the NDC was again electedand took office in January 200923.Notwithstanding these transfers of power in Ghana and in Malawi, beingan incumbent candidate does appear to bestow a significant advantage: theincumbent was the winner of the presidential elections in 1996 and 2004 inGhana and in 1999 and 2009 in Malawi. Where the incumbent candidate hashad to change – due to the two-term rule that applies in both countries24, the22 It is notable, however, that Freedom House judge that political and civil rights aremore restricted in Burkina Faso than in Ghana or Malawi. See Table 3 above.23 It has been noted that the 2008 election provided a test of the country’s democraticstrength: in spite of a very close margin of votes between the two dominant parties,state institutions withstood significant tension, in particular the judiciary and theelectoral commission (Allsop et al, 2009).24 This two-term rule was introduced in Burkina Faso in 2005 but not with retrospec-tive effect. Thus, President Blaise Campaoré was allowed to contest and win the 2005and 2010 elections but, without a Constitutional change, will be unable to stand in the2015 elections.

advantage for the incumbent party seems more limited, which may partlyexplain the results of the December 2000 and December 2008 elections inGhana, although the incumbent UDF party did win in Malawi in 2004.Despite these imbalances in the respective electoral systems, the fact thattransfers of power have taken place is fundamentally important, illustratingthat the balance of political power is more easily subject to change in Ghanaand Malawi, than in Burkina Faso. This probably goes a long way to explainthe tendency towards “populist spending” which has been common in pre-election periods both in Ghana and Malawi. As we noted above, pre-electionspending created significant fiscal imbalances in Ghana in 2000 and 2008,and in Malawi in 2004 and 2009.The available political analyses for Malawi and Ghana25suggest that polit-ical competition is strong not only across parties but also within parties, withprominent individuals often creating factions, which vie for leadership andinfluence within the party. Prominence within the party appears to derivelargely from the ability to enlist the support of popular national and localleaders who can help to win votes at presidential and parliamentary elections,or of rich businessmen, who can help to finance election campaigns. Thepotential to attract such support seems to depend heavily on the ability tobestow patronage. This patronage is bestowed by the award (or the promise ofthe award) of positions in Government, or, or by giving access (or promisingaccess) to significant commercial opportunities. A simple illustration of theimportance of patronage in Malawi and Ghana, relative to Burkina Faso isthe discrepancy in the numbers of Ministers, Deputy Ministers and Membersof Parliament per head of population26. CSO representatives interviewed ineach country asserted that similar patterns could be found in analysing thenumbers of positions on the Boards of Government Committees, Para-statalsand Trust Funds.

The purpose of identifying these differences is not to suggest underlying dif-ferences in the capabilities of political leaders in Burkina Faso, Ghana andMalawi (although these may also be at play) but to demonstrate that, over theperiod of the evaluation, they have been operating under different politicalstructures, which in turn have generated different incentives. Put simply, thepolitical environment is more competitive in Malawi and Ghana and, in gen-eral, political leaders have to resort to more extensive use of patronage inorder to come out on top. We will see that this has made a major difference tothe ability of each of these countries to provide leadership to PFM reformsand a consistent commitment to their implementation.

Table 5. Funding of PFM Reform Inputs in the Case Study Countries2001–2010272829

BurKINA FASo

External Funding US $ 30–35of PFM Reform In- million27putsDomestic Funding US $ 30–35of PFM Reform In- millionputsApproximate Total US $ 60–70millionMain Areas of Con- All clusters, butcentrationespecially Finan-cial Mgt & report-ing, Budget prepa-ration, RevenueAdministration,and Procurement

GhANA

US $ 39million28US $ 20millionUS $ 60millionAll clusters, but es-pecially FinancialMgt & reporting,Budget prepara-tion, Revenue Ad-ministration, Pro-curement andInternal & ExternalAudit

mALAWI

US $ 38–45million29US $ 30millionUS $ 68–75millionAll clusters, butespecially Finan-cial Mgt & report-ing, Budget prepa-ration,Procurement, In-ternal Audit, andExternal Audit.

It is, of course, exceedingly difficult to estimate spending on PFM reform with anydegree of accuracy and these figures should be interpreted only as a general indica-tion of trends rather than as precise estimates. There are, firstly, problems in classify-ing “PFM reform spending”, particularly for Government recurrent spending butalso for multi-sector donor projects, providing support both to PFM reform and topublic sector reform as a whole. Secondly, it is extremely difficult to obtain complete27 Based on the financial information available on 13 major donor projects in support toPFM strengthening, the Burkina Faso country study estimated US $ 21.35 million hadbeen provided but noted that data was missing on several other projects. De Renzio(2011) estimates donor disbursements of US $ 25–20 million for 1997–2007.28 The Ghana Country Study estimates external disbursements of US $ 51.1 millionfor 1998–2010, which has simply been adjusted pro rata to reflect spending for the2001–2010 period.29 The Malawi Country Study estimates that budgeted commitments over the periodwere as high as US $ 104 million, while noting that actual expenditures fell substan-tially below these budgeted amounts. In the quantitative study, De Renzio et al (2010)estimate total disbursements of US $ 38 million, which we take as the lower estimate,given that budgeted amounts were so much higher than this.

and accurate data on donor spending. For many grant-funded technical assistance orinstitutional support projects, data is completely absent; and where it is provided –both for grants and for project loans provided by multi-lateral agencies, data is morecommonly available for budgeted commitments than for actual expenditures.Notwithstanding these caveats, there are some important observationswhich can be made with a degree of confidence: ll of the case study countries have overseen major programmes of PFMA reforms over the past ten years, with total spending amounting to some US$ 60 –70 million. omestic allocations to PFM reform have also been high, both for recur-D rent funding and for funding of specific projects. This is especially true inBurkina Faso and Malawi. The significant inflows of General Budget Sup-port in each country, noted in Table 3, would have facilitated increaseddomestic budget allocations but their use to support PFM reform was ineach case an exclusively internal decision, unrelated to any conditionality. oth Ghana and Malawi experienced fluctuations in the total level ofB annual spending on PFM reforms, whereas Burkina Faso saw a moresteady, continuous pattern of spending.

2.3 REFORM OuTCOMESIt is difficult to show the evolution of the status of PFM systems over the fullten years, because of the difficulties of identifying a valid baseline. The onlyexisting PFM assessments, which might provide a potential baseline, are thePFM Assessments & Action Plans prepared for the HIPC countries in 2001and 2004. The graph in Figure 3 is derived from Table 1 in De Renzio et al,2010 (p.16), which provides a comparison, for 11 indicators, of scores of PEFAmeasurements undertaken over 2006 and 2007, with HIPC AAP scores of2001 and 2004, following a methodology developed in De Renzio & Dorotin-sky (2007). Figure 3 updates the analysis to take account of the PEFA assess-ments undertaken in 2010 and 2011. (The 11 indicators may be scored from 1to 3, thus the minimum score is 11 and the maximum 33.)

The Inception Report expressed doubts both about the robustness of theHIPC AAP assessments undertaken in 2001 and 2004 and about their com-parability with the later PEFA scores. The AAP assessments were prepared byjoint World Bank – IMF teams, with the purpose of generating ‘snapshots’ ofthe quality of PFM systems in HIPC countries for presentation to their respec-tive Boards. It was never expected that these assessments would provide thebasis for more extensive diagnosis and analysis of PFM strengths and weak-nesses. The 2001 assessments are considered especially “rough and ready”,and many commentators have suggested that the 2001 HIPC AAP assessmentfor Ghana gave too low a score and that for Malawi too high a score.Two basic trends emerge, despite these significant caveats. Firstly, BurkinaFaso is the one country that seems to show consistent improvements in itsPFM systems over the decade, resulting in a PFM performance well above theaverage for Sub-Saharan Africa in 2010. Secondly, although the status ofPFM systems in Malawi and Ghana is probably better in 2010 than it was in2001, both countries have exhibited fluctuations in PFM system quality, rath-er than steady improvement.The PEFA methodology is generally acknowledged to provide a more con-sistent and reliable measure of the relative quality of PFM systems than theHIPC AAP methodology. In line with the evaluation framework explained inChapter 1, PFM outcomes have been assessed for 6 “clusters” of PFM func-tions, following a methodology developed by Andrews (2010) and also used inthe quantitative study (De Renzio et al., 2010)30. Table 6 summarises the datafor each case study country, based on the earliest available and latest availablePEFA assessments.

Figures 4, 5 and 6 show for each of the 6 clusters of PFM functions, the 2006or 2007 and 2010 or 2011 PEFA scores for each case study country, plottedagainst a box plot of the minimum value, first quartile, median, third quartileand maximum value scores for the 100 countries in the De Renzio et al (2010)sample used for the quantitative study. These 100 countries comprise low andmiddle-income countries from Africa, Asia, Eastern Europe, Latin America &the Caribbean and the South Pacific, which have undertaken at least onePEFA assessment over 2006 to 2010. The full list is included in Annex 2,alongside the year in which the assessment was carried out. Together thesegraphs provide a good overview of the outcomes of PFM reforms at the end ofthe period under evaluation. Results are summarised below.

Burkina FasoBurkina Faso’s PFM reform programme has been successful ingenerating improvements in most key aspects of public financemanagement.By 2010, PFM systems in 5 out of 6 clusters were scored atlevels on or above the third quartile of the sample of 100 countries, and theremaining one, External Accountability, was at the median level.Taking the period as a whole, the areas where the highest PEFA scoreswere achieved were in strategic budgeting, budget preparation, internal con-trol and accounting and reporting. Resource management is the area, whichsaw the largest improvement over the 2007 to 2010 period, as a consequenceof the introduction of long delayed tax reforms, as well as improvements indebt and cash management and procurement systems. Overall, these fivePFM “clusters” are the ones, where PFM reforms have advanced fastest.These reforms have embodied both ‘de iure’ and ‘de facto’ reforms, as well asreforms undertaken by concentrated actors (revenue reforms) and deconcen-trated actors (accounting & reporting, procurement, budget preparation.)Improvements in external scrutiny and audit have been considerably slow-er, showing in fact a modest decline between 2007 and 2010. Although exter-nal audit has been an area of attention, following the creation of the Cour desComptes in 2002, it has not been an area of high reform spending, in partbecause the difficulties of recruiting adequately qualified staff for the CdChave placed a limit on its pace of growth and, by implication, on the level ofcapacity building assistance which could be effectively provided. Legislativescrutiny, the other aspect of this “cluster,” has not been a major focus of atten-tion of PFM reforms, and is one of only two areas (the other being the timeli-ness of budget support disbursements), where PEFA scores deterioratedbetween 2007 and 2010.PFM reform efforts by the Executive have been effective in generatingimprovements in PFM system outcomes but it seems that there is limitedimpetus for PFM improvements from sources outside of the Executive. Thequality of public access to fiscal information has shown no change at allbetween 2007 and 2010. Similarly, the timeliness and the thoroughness of theLegislature’s examination of the report of the Cour des Comptes on the Loi deRèglement (final accounts) have actually deteriorated. Thus, the Executive hasbeen able to direct a well-structured and effective programme of PFMreforms, and to attract external funding in support of this programme, butother actors – notably civil society and the Legislature – are yet to engageeffectively with the aspects of the PFM reform agenda, where their influencemight be expected to be significant.

was also made with budget preparation and resource management31. Butscores for internal control & audit and external accountability deteriorated,and there was no improvement in the poor quality of accounting and report-ing. In particular, there were continued problems and delays with accountsreconciliation, in-year budget reporting and annual financial reporting. Lackof progress in these areas may be largely explained by the failure of theBPEMS integrated financial management system.Relative to the significant funds expended on PFM reform over the period,progress has been disappointing. The most substantial success has been instrengthening the legislative base but the Government has experienced signifi-cant challenges in implementing the new laws. Otherwise, the most effectivereforms appear to have been the revenue management activities, as they haveled to a sustained output in the form of changed processes (successful introduc-tion of VAT, and the introduction of the TIN), and there has been a signifi-cant increase in revenues as a share of GDP during the period studied.Reform initiatives requiring actions by large numbers of deconcentratedactors have been less successful than initiatives, under the responsibility ofsmall groups of concentrated actors. For example, improvements in revenuemanagement were largely the result of actions within the revenue manage-ment agencies and witnessed continuous progress over the period. By contrast,the two major reforms introduced under PUFMARP – the PFM reform pro-gramme spanning the period 1997–2003 – involving the introduction ofa more policy-led medium-term budget process (the MTEF) and the introduc-tion of an integrated financial management system (BPEMS) failed to embedthese improved practices and procedures across the central government.Despite ample resources, these projects have not contributed to improvedintermediate outcomes, nor prevented the deterioration in such outcomes.

MalawiDuring aconcentrated period from 2005 to 2008, Malawi wasable to achieve significant improvements in its PFM system out-comes, but was not able to achieve consistent progress over thedecade.By 2011, the quality of Malawi’s PFM functions in 4 out of 6 clusterswas above the median of 100 countries, and average scores within each of the6 clusters improved from 2006 to 2011. However, outside of certain importantlegislative changes, the period 2001 to 2004 was characterised by a decline inPFM performance. Over 2009–2011, signs of deterioration have againemerged.Over 2006 to 2011, progress in strategic budgeting was driven by improve-ments in systems to assess debt sustainability and by improved costing of sec-tor strategies. Resource management was also significantly strengthened, dueto improvements in revenue administration, debt management, cash manage-31 A number of the improvements in outcomes were independent of the main PFMreform actions: e.g., the passage of the Appropriations Act before the end of the fiscalyear improved the budget preparation process and the completion of a new DebtSustainability Assessment (under IMF leadership) contributed to the improvement inStrategic budgeting.

ment, procurement and payroll management. For all of these, one can tracka line from reform inputs to achieved outputs, to the improved intermediateoutcomes. Similarly, the (more modest) improvements recorded in internalcontrol, audit and monitoring and in accounting and reporting can be relateddirectly to investments in the integrated financial management system(IFMIS), and in the Internal Auditing system.However, there was also a PEFA assessment in 2008 and, when the relativechanges in sub-dimension scores from 2006 to 2008 and 2008 to 2011 arecompared, it appears that most of the improvement occurred prior to 2008,and that there was more deterioration between 2008 and 2011 than between2006 and 2008. Between 2006 and 2008, 27 scores improved (equivalent to45 % of scores that could improve; in other words were not an A already).Between 2008 and 2011, only 7 scores improved (15 % of the scores that couldimprove). Moreover, 35 % (or 20) of the scores that could deteriorate did so inthe second period, compared to 17 % (or 8) in the first period.The period prior to 2005 did include ‘de iure’ improvements in the form oflegislative changes – notably, the approval of the Public Financial Manage-ment Act and the Public Procurement Act in 2003. However, the HIPC AAPassessments of 2001 and 2004 suggest that, overall, it was a period of declinein the quality of PFM systems.

2.4 ThE PFM REFORM PROCESSES inOvERviEW: ThE RESulTinG C-M-OCOMBinATiOnSWe have used a “realist synthesis” approach (Pawson & Tilley, 1997; Pawson2002) to address the higher-level questions posed in the terms of reference. Asdescribed in Chapter 1, this has involved a careful analysis of the combina-tions of Contexts (C), Mechanisms (M) and Outcomes (O) thrown up by thethree country case studies and the nine case histories within them. The fol-lowing emerge as the factors, which have been critical in determining the suc-cess or failure of reforms: nBurkina Faso,the deep, consistent and technically informed nature ofI political commitment to reform has been a fundamentally important con-textual factor. PFM reform has been a high priority for Government sincethe mid-1990s and much of the groundwork for the improvements noted inthe past decade were laid in the late 1990s. The incentives for careful plan-ning and monitoring of PFM reform created by the Budget Supportarrangements introduced in 2003 have also been important, as have theideas and incentives provided by the WAEMU Commission. Budget Sup-port funding has also facilitated the funding of PFM reform actions fromthe National Budget. On the negative side, the policy space for reform hassometimes been unnecessarily closed and the Development Agencies havenot always been able to bring to the table an adequate knowledge andexperience of PFM reform, most particularly with regard to the sequencingof budgeting reforms, involving programme budgeting and medium term

expenditure frameworks. Regarding mechanisms, two fundamental factorshave been the establishment of a strong framework (the SP-PPF) for coordi-nating the implementation of reforms, and the development of a structuredframework for channelling donor support to the programme, monitoringprogress and managing dialogue with external and domestic stakeholders(the PRGB/ SRFP). nGhana,the equivalent level of political commitment to reform has beenI absent, even though many core reforms were also initiated in the late 1990s.There has been no overt opposition to reform at senior political levels butpolitical support has been distant, and rather passive, with PFM reformperceived as a technical, not a political issue. Thus, political leaders haveneither taken care to ensure the appointment of good technical leaders forreforms, nor to intervene to correct design mistakes and to resolve the civilservice “turf battles” which have periodically blocked reforms. Ghana hasalso suffered policy space constraints: in particular, its Development Partnersdid not offer an adequate range of ideas on international practise and onunderlying change management issues. This had a major negative impact onthe development of sustainable systems for multi-year budgeting (the MTEF)and for integrated financial management and accounting (BPEMS). In rela-tion to mechanisms, Ghana’s PFM reform plans were fragmented and notsystematically updated. Its coordination structures were also fragmented anddid not have a sufficient level of authority to intervene to correct implementa-tion problems, or to learn from mistakes. Although PFM reform constituteda significant part of the policy dialogue under Multi-Donor Budget Support(MDBS), these dialogue structures also proved ineffective as a framework foridentifying and correcting mistakes in PFM reform design and implementa-tion.

M alawienjoyed strong political support for the implementation of PFM reform over 2004–2009, as is evidenced by the nomination of strong tech-nicians as Vice President and Minister of Finance, and the sourcing of sig-nificant domestic funding for PFM reform. But this support was not presentat the beginning of the decade and has again weakened since 2009. A con-tributory factor in this pattern has been the significant, pro-reform influ-ence enjoyed by the Legislature over 2004–2009, the only period in thedecade when the Government did not have an overwhelming majority inParliament. While Development Partners helped to establish the mecha-nisms to plan and coordinate reforms and provided harmonised financialsupport, they were not quick enough to perceive the window of opportunityoffered in the 2004–2009 period. As a result, Malawi is the only country ofthe three to have suffered a financial constraint, where the pace of reformwas significantly hindered by a lack of finance. Malawi also suffered policyspace constraints, with a number of “best practise” reforms being pushedupon it, which were not appropriate to its institutional context, especiallyin the areas of procurement and internal audit. Malawi’s coordinationstructures were reasonably effective, even if the underlying plans were notalways adequately prioritised.50

Further nuance is added by the analysis of the 3 case histories within each ofthe three case study countries. Here, we provide the highlights, which illus-trate the most important underlying trends: nBurkina Faso,the three case histories show one example of criticalI success factors all coinciding to produce a case history of outstanding suc-cess, one example of success constrained by limitations on policy space andone example of politically constrained success: The introduction of theCircuit Intégré de la Dépense (CID)and the progressive computerisation of the finan-cial management systemis an example of a reform, whose sim-ple design and gradualist approach to implementation were excel-lently adapted to the institutional and capacity context. The processstarted with the internal development (by 4 government computerprogrammers working alongside two long-term consultant program-mers, financed by the World Bank) and introduction in 1996 of theCIDmodule which integrates all steps of the expenditure process,from budget preparation to execution. Accounting and revenue mod-ules followed (CIE and CIR) both of which were operational from2000, and other modules have been subsequently added. Apart fromthe home-grown, gradualist approach to development, another fea-ture has been a pragmatic approach to the degree of integration ofthe system. Instead of having all systems fully integrated in real time,which would demand a high degree and reliability of inter-connec-tivity, connections between most modules of the system are managedon a periodic basis (some nightly, some weekly) using a software tool,calledi-bus.Whilst the development of the system has suffered occa-sional set-backs and problems32, the end result has been a relativelylow cost, integrated financial management system, managed directlyby theDirection des Services Informatiquesof the Ministry of Finance,using systems appropriate to the needs and to the telecommunica-tions infrastructure33. The experience with programme budgeting (budgets-pro- gramme– BP) and with the related processes of sectoraland global CDMTs (MTEFs)forms a sharp contrast to that ofthe CID and the computerisation process. Following a visit to Cana-da in 1997, the then Minister of Finance became interested in pro-gramme budgeting as a way of linking public spending to develop-ment results. With the support of the World Bank,budgets-programmes32 Notably, the initial team who developed the CID departed prematurely and had tobe replaced unexpectedly. In addition, the initial software used for the CID becameobsolete and had to be upgraded over 2004/ 05.33 The centralized system of public finance management, which has prevailed in Burki-na Faso over the evaluation period, has meant that the number of required users ofthe system has been relatively small. As responsibility for commitment of expenditure(engagement)and authorization of payment(ordonnancement)become deconcentrated,more users will need to be brought onto the system, which will have implications forhardware, for interconnectivity and for training.

were launched in 6 pilot ministries in 1999 and extended to all min-istries in 2000. However, the objectives were misunderstood by sec-tor ministries, who used budgets-programmes as a method todemand additional resources rather than as a method for more effi-cient and transparent programming of existing resources. With theintroduction in 2001 of the CDMT-Global as a tool to determineaggregate fiscal targets and define ceilings for budget preparation,the problem of using budgets-programmes as “wish lists” wasresolved, but the fundamental purpose of the exercise remainedunclear to most sector ministries. Given the evident lack of influenceof the budgets-programmes, from 2004 onwards a number of donorspushed for the introduction of sectoral MTEFs(CDMT sectoriels).Unfortunately their introduction simply duplicated the BP processand caused further confusion at the sector ministry level. In 2008,a committee was established to review the experience and developa revised approach to programme budgeting. It was led by the Min-istry of Finance but included members of sector ministries and alsodrew on the advice of IMF AfriTAC. The Committee recognised theneed to develop comprehensive guidance material. A small team ofgovernment staff was created to lead this process, working during2010 with seven pilot ministries. In 2011 a new strategy documentwas prepared as well as two detailed methodological guides, and theprocess was extended to 16 pilot ministries. Thus, a structured andsequenced process to the introduction of programme budgeting,which looks appropriate to the institutional needs and capabilities,has now been adopted and is under implementation. Yet, the initialreforms led to 9 years of largely wasted efforts because of the failureto appreciate the complexity of programme budgets and the particu-lar sequencing problems involved. The Government looked to itsDevelopment Partners for advice on these issues, and up until 2008the response was fundamentally inadequate. reform of Revenue Administration in Burkina FasoisThethe one area where there has been doubt over the degree of politicalcommitment to reform. Certainly, in terms of achieving the essentialobjective of increasing the proportion of GDP collected as domesticrevenue, these reforms have been less successful than others andthere have been several World Bank and IMF reports pointing toa lack of government commitment to revenue reforms. A more care-ful analysis shows that there has been political support for certainreforms but not for others. There is a clear concern within Govern-ment over the political costs of removing exonerations on VAT items,as well as on the agriculture and mining sectors. By and large, theseexonerations have not been removed, despite the pressure that hasbeen exerted by donors to have them removed. On the other hand,significant improvements have been made in the quality and efficien-cy of revenue administration, including the introduction of a singletax identification number(Identifiant Financier Unique – IFU),the

establishment of a large taxpayers division(Direction des Grandes Entre-prises),the successful introduction of three inter-linked computer sys-tems (SYDONIA, SINTAX and CIR), and a major reform of taxlegislation (in 2010). These reforms could not have been introducedand sustained without an internal commitment to these reforms – atboth political and administrative levels. Thus, the Government hasbeen committed to modernising and improving the efficiency of taxadministration but it has chosen, for political reasons, to proceedslowly with the expansion of the tax base. A failure on the part of theDevelopment Agencies to appreciate and accept this has created anunnecessarily high level of transaction costs in the management ofthe revenue reform programme, as well as inefficiency in the provi-sion of external technical assistance. nGhana,two case histories highlight the problems of inadequate policyI thinking on PFM reform but also the difficulty of learning from mistakesand adapting reform designs in a context where reform coordination isweak and political leadership largely absent. The third case shows thatgood progress can sometimes be made in such contexts, where there ispolitical commitment, good technical leadership and an adaptive, flexibleapproach to implementation: Budget Planning & Expenditure Management SystemThe(BPEMS)was introduced under the PUFMARP programme in1997. It was intended as an integrated system to manage the fullcycle of expenditure from budget preparation, through execution tothe preparation of accounts. It was also intended to be comprehen-sive, eventually covering all central government expenditure. Thechosen system was Oracle Financials, a system designed for the cor-porate market, which it was understood would require substantialcustomisation. In retrospect, the project design was criticised forbeing technology-driven, and more specifically for giving insufficientattention to change management issues, and to the assessment ofcapacity constraints and training needs. It was also criticised foradopting an ambitious “big bang reform” approach rather thana more gradual, “incrementalist” approach. In these respects,BPEMS suffered from a significant “policy space constraint”: laterinternationally funded IFMIS initiatives in other parts of Africaaddressed change management, capacity and training needs muchmore explicitly and exhaustively but Ghana perhaps suffered frombeing a pioneer of such projects34. Yet, what is more striking is whychanges were not introduced, why did the project management team34 Fyson (2009) presents cogent arguments to suggest that this inappropriate projectdesign was not simply an imposition by the World Bank funders and their consultants:the “state of the art” model was one to which senior Government officials stronglyaspired, to the extent that they opposed simplification of the design when this wasfirst mooted by the World Bank in 1999. We examine these arguments more fully inChapter 3 below.

not learn from mistakes? The simple answer is that there were funda-mental weaknesses in the structure of the project management team,which prevented this happening. BPEMS was initially managedthrough the PUFMARP project implementation unit, who had nei-ther functional responsibility for budget execution and accountingprocesses nor operational responsibility for reform implementation,which was sub-contracted to different sets of consultants. This result-ed in the project being distanced from its principal client depart-ments (CAGD and the Budget Department) but also in a projectmanagement framework where accountability was blurred and therole of government distinctly limited, especially because the PUF-MARP PIU itself comprised of consultants rather than governmentstaff35. These problems were compounded by the lack of clear politi-cal leadership over the process. Although in the late 1990‘s the NDCGovernment had expressed support for PUFMARP as a whole andfor IFMIS in particular, they were content to retain an arm’s lengthrelationship to its management36, rather than ensuring that a clearchampion (such as the Accountant General) was nominated to leadthe process. When the government changed after the 2000 elections,the incoming NPP Government introduced new staff throughout thehierarchy of the Ministry of Finance which brought major disrup-tions and delays, but the new government also failed to nominatea clear government champion to lead the process. The project man-agement structure thus remained unchanged until PUFMARP end-ed in 2003, when responsibilities were formally transferred toa BPEMS secretariat placed within the CAGD. By this stage, thelack of progress in the development and roll-out of the system had ledto the withdrawal of external funding. Although Government tookover funding at this stage, the underlying design, management andleadership problems were never satisfactorily resolved. By 2010,despite an investment outlay in excess of US $ 20 million, the systemwas not operational in any of the 8 pilot MDAs. Moreover, much ofthe software and hardware had by this stage become obsolete. Withthe change of government after the December 2008 elections, a fullreview of the project was undertaken in collaboration with the WorldBank, and a new $55 million project to install a new financial man-agement system (GIFMIS) was agreed in 2011. hana’s Medium Term Expenditure FrameworkwasGanother pioneering venture for PFM reform in Africa. In common35 A Donor official quoted in Fyson (2009) said that, “The PMU had no authority – itwas like an island by itself. The consultants report to the Minister, the Deputy &the Chief Director but in a real sense they don’t have any authority to do anythingbecause they are all consultants.”36 The PUFMARP newsletter of 1997 explained the management responsibilities asfollows: “consultants are responsible for developing and implementing the variouscomponents, while at the same time transferring skills to Project ImplementationTeams, comprising of Government of Ghana counterparts.”

with BPEMS, it also suffered from a “policy space” constraint, beingmodelled on an idiosyncratic vision of an MTEF, which did not takeadequate account of wider international experiences nor of the dif-ficulties of sequencing and reform coherence which were likely to bethrown up by the Ghanaian context. In a narrow sense, the outputsof the MTEF reforms, measured against the initial project specifica-tion, were to a large extent met, and after initial donor funding fin-ished have been subsequently sustained. However, measured againstthe requirement to establish a more realistic and strategic, mediumterm budgeting process, the targeted outcomes are a long way fromhaving been achieved. The reform design was idiosyncratic in tworespects: firstly, it placed emphasis on the “bottom-up” elements ofan MTEF (programme-based budgeting, detailed costing of plans,integration of aid-financed projects) and barely mentioned theupstream elements of an MTEF (macro-fiscal framework, strategicpolicy and expenditure review processes) that are conventionallyseen as the initial priorities in establishing a robust MTEF process.Secondly, it adopted an activity-based budgeting approach to thedevelopment of multi-year budgets, which generated voluminous,highly detailed documents, which made it difficult to discern thestrategies and priorities underlying MDA budgets, thus effectivelydefeating the purpose of the exercise. Whilst concerns over the direc-tion of the MTEF were raised in annual reviews and most explicitlyin the PUFMARP Implementation Completion Report, theapproach remained unchanged until very recently with the decisionin 2010 to replace activity-based budgeting with a more strategic,programme-based budgeting approach. Why this persistence withan MTEF model which was not delivering a strategic approach tobudgeting? Again, the answers relate predominantly to the mecha-nisms chosen for project management and the overall political con-text for the reforms. The MTEF reform operated as a separate sub-project within PUFMARP, under an MTEF Project Unit, housedinside the Budget Department of MoFEP. Because funding was pro-vided directly by DFID, the MTEF component operated relativelyindependently of the PUFMARP PIU. This degree of autonomy washelpful in generating strong ownership for the reforms within theBudget Department but undermined coordination with otherdepartments of MoFEP and other components of PUFMARP. Forexample, the Economic Policy & Forecasting Division of MoFEPshould have played a much stronger role in the MTEF reforms.Linkages with BPEMS should also have been promoted. Yet, themost significant weakness of the management arrangement was that,in common with BPEMS, it left the primary responsibilities forreform design and implementation in the hands of consultants.Again, in the absence of a political impetus for the MTEF, seniordecision-makers did not intervene to change this.55

 hana’s Revenue Administration reformswere introducedGunder PUFMARP 1997–2003, under DFID and World Bank fund-ing and continued under the support to the Revenue Agencies Gov-erning Board (RAGB – later the GRA: Ghana Revenue Authority)provided by GTZ/ GIZ from 2003. Without a doubt, these reformscomprised the most successful component of the overall PFM reformprogramme. They permitted the re-introduction of VAT in 1998(following the initial failed attempt in 1995) and the progressiveincrease in rates from 10 % to 12.5 % and subsequently to 17.5 %(with the inclusion of the National Health Insurance Levy and theGhana Education Trust Fund Levy), as well as the introduction ofthe Tax Identification Number (TIN), the creation of the Large Tax-payers Unit (within GRA), the creation of the Tax Policy Unit (with-in MoFEP), the computerisation of internal management systems,and the initiation of the merging of the three revenue departmentsinto the Ghana Revenue Authority. Together, these reforms havepermitted a significant, and sustained increase in tax collections asa percentage of GDP, whilst also reducing administrative costs andimproving the rate of collection of tax arrears. There are a numberof factors, which seem to have been critical to these successes. Firstly,there has been a strong political interest in raising tax revenues, illus-trated by the active engagement of two Presidents (Rawlings andKuffour) as well as Government Ministers and Parliamentarians ofboth NDC and NPP in the process of re-introducing VAT, in thesubsequent rise in rates, and the subsequent incorporation of theNational Health Insurance and Education Trust Fund levies into theVAT process. There has also been political support to the process ofcreation of the GRA, which ensured that the in-fighting between the3 former revenue departments became a temporary rather thana permanent obstacle. Secondly, reform management structureswere directly integrated within the appropriate government struc-tures. Thus, there was no specific “revenue reform unit”, independ-ent of the RAGB/ GRA. Thirdly, the approach to the design andimplementation of reforms has been incremental and responsive tocircumstances, creating a “learning approach” to implementation.An example was the approach to the re-introduction of VAT in1998, which sought very deliberately to correct the mistakes madeduring the failed launch of 1995. More recently, GIZ’s assistance tothe process has been structured in 3-year tranches, incorporatinga participatory review of past progress, leading to joint developmentof the new 3-year programme. nMalawi,two case histories show the difficulties of applying “interna-I tional best practice” in contexts of weak organisations and scarce technicalskills, while the third case illustrates that institutional, financial and initialdesign constraints can be overcome where there is strong political andtechnical leadership:56

 alawi’s Procurement reformsprovide an excellent exampleMof “international best practice”, gone wrong. In 2003, the GoMenacted the Public Procurement Act, which in essence createda decentralised procurement system. It provided for decentralisationof public procurement decisions and responsibility to procuring enti-ties, the establishment of a procurement cadre as a new professionalstream within the civil service, a new set of procurement methods,and the creation of the Office of the Director of Public Procurement(ODPP) as the body responsible for regulation and monitoring. Bymid-2011, most of the architecture of the new procurement systemwas in place, as a result of the reform outputs, generated with domes-tic and external funding. However, for the most part, the procure-ment system in 2011 remained non-functional. The 2011 PEFAassessment scored it as a D+. The 2009 Compliance assessmentundertaken by the GoM found that: while entities had procuremententities, these were often recreated stores units without the necessaryprocurement skills; not all members of procuring units had been ori-entated on the regulations governing procurement and where theyhad, they did not put into practice what they had learnt; there wereseveral problems with the functioning of the IPCs, including that insome cases the controlling officer still had final decision-makingpower; and the ODPP itself faced significant staff constraints. Inshort, the significant shortages of trained staff within the civil ser-vice, combined with the persistence of hierarchical modes of work-ing, limited accountability and a culture of frequent disregard ofrules had made it impossible to implement the ‘best practice’ modelof decentralised procurement, despite the extensive financial outlays. modernisation of the Internal Audit functionhas beenTheanother important component of Malawi’s PFM reforms, supportedby several Development Agencies. Again, its success has been con-strained by the inappropriateness of the models, which dominatedthe reform agenda (a “policy space” constraint) and by the inconsist-ency of political support. The Internal Audit Act of 2003 sought toestablish a comprehensive internal audit service, which would reportto the controlling officers of each ministry, whilst respecting a stand-ard set of norms for the conduct of audit work and a risk-basedapproach to the programming of activities. The need for a modernrisk-based internal audit function was acknowledged by the GoM inPFEM planning documents. Yet, the roll out of reform activities andoutputs beyond establishing the Central Internal Audit Office, Inter-nal Audit units and committees in ministries, departments and agen-cies, has been funded if not driven by donors, without much financialinput by the GoM beyond covering the recurrent costs of internalaudit units. As of 2011, key enabling documents such as the InternalAudit Guidelines and Charter are ready for implementation, but for-mal sign-off by the GoM has been pending for some time. However,the limited political support for Internal Audit reforms is perhaps not

the most important constraint to their implementation. In the face ofdramatic staff shortages, it is difficult to see how a decentralisedinternal audit function might work. At the end of 2010 the establish-ment for internal audit services was 129, out of which 99 posts werefilled. However, the Government’s functional review recommended364 posts for the service to function effectively. This would mean 265vacant posts for Internal Auditors. In short, there remains a funda-mental policy problem, which has still not been adequatelyaddressed. development and installation of the Malawi IFMISTheover 2005–2009is a striking success story, which powerfully illus-trates the importance of political will and good project management.The GoM’s first effort to develop an IFMIS started in 1996. By2004, at least US $ 8 million had been spent but a 2004 review foundthat while outputs were in place, these outputs in combination didnot add up to a functioning system. After the change of government,the incoming President Mutharika put control over governmentspending at the centre of an economic governance reform pro-gramme. This meant putting in place a functional financial manage-ment system, which would end controlling officers’ discretion tospend beyond budgeted appropriations, and would give the centrecontinuous access to reliable information on spending. A newAccountant General was appointed (the current Auditor General),with a mandate to do whatever was necessary to get the system inplace. By May 2005, it was decided that the 21 issues identified in the2004 review as essential to rescue the existing Coda-based systemcould not be resolved and that the GoM would replicate an Epicor-based system from Tanzania, where the system was already in place.It was also decided to recentralise payments using the IFMIS. Fivelinked bank accounts were opened at the central bank and 150 com-mercial bank accounts were closed. Thereafter, the core IFMIS out-puts were delivered within 18 months of the 2004 election, and rolledout to all national ministries within 26 months, on a completely newplatform. This was after 8 years of previous investment that had notresulted in a functional combination of outputs. Respondents arguedthat the reason for the focus was the urgency assigned by the centralpolitical leadership to gain control over expenditure. Fundamentalto the whole success was the fast replacement of the former Account-ant General with an action-orientated and experienced PFM person,whose actions were backed by the Secretary to the Treasury, theMinister of Finance and the President himself. In addition, US $14million of government funding was provided for the process. A 2007East Afritac Review of the system found that “implementation of theEpicor system, to date, has been an impressive achievement particu-larly when viewed against similar international experience”. In addi-tion to political commitment, it acknowledged that much of the suc-cess could be attributed to a streamlined project management

3. Successful PFM reform:what is the right context?3.1 inTROduCTiOnThis chapter examines in detail the implications of our findings regarding thecontext for successful PFM reform. Specifically, we ask what have been thecritical factors driving or blocking change in each of the study countries. Westart by considering the results as a whole, evaluating their implications for theprogramme theory underpinning the evaluation framework. We then focus onthe two factors which stand out as being of critical importance in creatinga context for successful PFM reform, firstly the issue of political commitmentand leadership, and secondly the question of policy space: the depth, breadthand suitability of the menu of ideas which shape the design and implementa-tion of PFM reform.An overview of the C-M-O (Context-Mechanism-Outcome) combinationsidentified from the nine case histories analysed at the country level is present-ed in Table 7. The table draws from the more extensive analyses presented inthe country studies themselves. The previous chapter provided a summary ofthe results of the analysis, but the more detailed analysis in Table 7 is includedhere as a key point of reference for the conclusions we have reached regardingthe most important contextual factors for successful PFM reform.

Table 7. Overview of PFM Reform in the Case Study Countries:the C-M-O Combinations

CoNtExt

BurKINA EQ 8:Political commit-FASoment has been (i) con-sistent and long-stand-ing, (ii) deep, and (iii)technically informed. Fi-nancing has been suffi-cient. Some limitationson policy space, esp re.Programme Budgets.EQ 3:Limited & incon-sistent use of countrysystems.Budget Support 31 % ofODA, 12 % of publicspending, but unpredict-able within-year dis-bursements. BS dialoguehelped to institutionalisereform monitoring pro-cess but conditionalitynot influential.EQ 4:Civil Society & Leg-islature uninfluential butWAEMU Commissionasignificant influencethrough peer pressure &directives on PFM.

mEChANISmS

EQ 2:strong coordinationarrangements, based uponan integrated reform pro-gramme (PRGB/ SRFP),endorsed at Cabinet level,led by Secretariat (SP-PPF)of high calibre staff, report-ing to Minister of Finance,with harmonised frame-work for support to re-forms.EQ 5:Reform inputs rele-vant in targeting identifiedareas of weakness, ad-dressing reform issues ofinterest to the politicalleadership, and in adaptingto institutional constraints.Active learning and adapta-tion process.EQ 7:Most outputs inGoBF’s PFM reform pro-gramme (PRGB/ SRFP) de-livered, without excessivedelays. TA not linked to out-puts in PRGB/ SRFP con-siderably less efficient. Re-form sequencing generallygood, with gradual devel-opment of integrated FMsystem exemplary. Se-quencing of programmebudgets/ MTEF reformswas wrong but correctedlate in period (2008–2011).

outComES

Burkina Faso’s PFm re-form programme hasbeen successful in gen-erating improvementsin every key aspect ofpublic finance manage-ment.By 2010, PFMsystems in 5 out of 6clusters were scored atlevels on or above thethird quartile of thesample of 100 countries,and the remaining one,External accountability,was at the median level.

CoNtExt

GhANAEQ 8:Despite initial en-thusiasm for PUFMARP,political commitment toPFM reform was neverdeep (PFM reforms seenas “technical”) and fluctu-ated over the electoral cy-cle. As apioneer of MTEF& IFMS, Ghana sufferedfrom limited policy ideas,which underestimatedreform complexity/change management im-plications. Financing suf-ficient.EQ 3:Limited & incon-sistent use of countrysystems.Budget Support 29 % ofODA, 9 % of public spend-ing, with improving pre-dictability of disburse-ments 2004–2009. BSdialogue helped to im-prove reform monitoring+ level of debate butfailed to solve big prob-lems with BPEMS &MTEF. PFM “trigger”conditions not influential.EQ 4:Limited domesticpressure for PFM reformbut opening to public ofPAC hearings has led topressure on PAC to in-crease its technical ca-pacities. Regional peer-to-peerexperience-sharing citedas important in enablingGoG officials to accessInt. experience, e.g.CABRI.

mEChANISmS

EQ 2:Two reform pro-grammes: PUFMARP1997–2003 & ST/MTAP2006–2009 with 3 year gap.Neither plan was system-atically updated on annualbasis. Revenue & Audit re-forms managed outside ofthese frameworks. PUF-MARP reported to DeputyMinister of Finance & hadaproject implementationteam, later converted intoBudget Devpt Unit withinBudget Division.EQ 5:Reform inputs wererelevant in targeting identi-fied areas of weakness butBPEMS & MTEF not welladapted to technical & sys-tem constraints & did notaddress change manage-ment. Overall, learning andadaptation very slow.EQ 7:Outputs on BPEMSnever delivered in function-al form, & IPPD2 deliveredwith 4-year delay, with in-complete coverage. Pro-curement reforms nevercompleted. MTEF outputsdelivered but inappropriateto end objective. Internal &External Audit reform out-puts delivered but im-provement in functionalitystill modest. Revenue re-forms successfully & ef-ficiently delivered.

outComES

Ghana showed only lim-ited improvements inPFm intermediate out-comes over the period.Even though PFM re-form spending was con-centrated in 2000–2006,in 5 out of 6 clusters,PEFA scores in 2006were worse than themedian of 100 countries.Over 2006 to 2010, im-provements in strategicbudgeting also brought itabove the median butscores in 2 other clus-ters deteriorated, andthere was no improve-ment in the poor qualityof accounting and re-porting.Relative to the signifi-cant funds expended onPFM reform over the pe-riod, progress has beenpoor. Most substantialsuccess has been instrengthening the legis-lative base but the Gov-ernment has experi-enced significantchallenges in imple-menting the new laws.Otherwise, most effec-tive reforms have beenin revenue manage-ment, which have led tothe successful introduc-tion of VAT, and the intro-duction of the TIN, andasignificant increase inrevenues as ashare ofGDP.

CoNtExt

mALAWIEQ 8:Extent and natureof political constraintshas varied by reformarea and by reform pe-riod In some PFM func-tions (e.g. Internal Au-dit), lack of politicalsupport has resulted inan under-supply of nec-essary outputs. In oth-ers such as IFMIS, politi-cal support has ensuredthe minimum level ofoutputs necessary forPFM functionality. Therehas been ahigh focus onputting in place laws,rules, systems and pro-cedures that follow in-ternational best prac-tice, and alack ofconsideration for theskill and the organisa-tional change require-ments for their imple-mentation. Thisover-emphasis on inter-national best practice isinterpreted as apolicyspace constraint.Over 2004–2007, pro-duction of reform out-puts was slowed downby financial constraints.Arguably, donors did notstep up their supportsufficiently in the 2004to 2009 reform window.EQ 3:Limited & incon-sistent use of countrysystems. Increasing useof SWAp arrangements.Budget Support

mEChANISmS

EQ 2:Establishment of for-mal structures to coordi-nate PFM reforms (PFEMsteering committee, tech-nical committee and unit,and various workinggroups) was important increating capacity to sup-port ownership and leader-ship of reform plans. TheGFEM group– and CABS–provided astructure toharmonise donor PFM in-puts.However, until 2010, thePFEM Action Plan was anamalgam of individual re-form interests, not acoordi-nated and sequenced re-sponse to PFM weaknesses.EQ 5:Reform inputs weredirected at PFM weakness-es, but PFM reform modelsnot sufficiently adapted tothe institutional context.Reforms in procurement,internal audit and budget-ing followed internationalbest practice, but were notsuited to acontext in whichtechnical and managerialskills were scarce, and dif-ficult to recruit or to retainwhen trained. Less sophis-ticated reforms might havegenerated greater im-provements in functional-ity.EQ 7:Up to 2004, reformswere inefficient on accountof low reform output com-pared to targets and inputs.Over 2004–2009, consistent

outComES

During aconcentratedperiod from 2005 to2008, malawi achievedsignificant improve-ments in PFm out-comes, but was not ableto achieve consistentprogress over the dec-ade.By 2011, the qualityof Malawi’s PFM func-tions in 4 out of 6 clus-ters was above the me-dian of 100 countries,and average scoreswithin each of the 6 clus-ters improved from 2006to 2011. However, apartfrom legislative chang-es, 2001to 2004 sawadecline in PFM sys-tems. Over 2009–2011,signs of deteriorationhave again emerged.Nevertheless, major re-form outputs were de-livered over the period,signalling significantchange in the capacityfor budget preparation,budget execution andaudit. Most outputs putin place had externalsupport but the effect ofthe political change in2004 and the degree towhich it drove reformsfrom within is signifi-cant. In retrospect,among the reformachievements of theGoM, the procurementand roll-out of an IFMIS63

CoNtExt

increasingly importantover period reaching 40 %of ODA in 2008, before be-ing suspended by DFID andother agencies over2010–2011. BS dialoguehelped to formalise thePFM reform action planand had aheavy influenceon its content, but PFM“trigger” conditions not in-fluential.EQ 4:Domestic pressurefor reform played arole inputting procurement re-form and effective exter-nal audit on the reformagenda. Pressure of theLegislature also consid-ered important in Presi-dent Mutharika’s 1sttermin sustaining pressure forreform. Political impetusfor PFM reforms is said tohave weakened in 2ndtermwith amajority in Parlia-ment and no 3thterm tocontest. Learning from re-gional experiences and in-ternational standards hasbroadened the policyspace for reforms.

3.2 REFininG OuR PROGRAMME ThEORyThe ‘programme theory’ underlying the evaluation framework is explained inChapter 1, where it is illustrated in Figure1 and, in more simplified form, inFigure 2. The theory postulates that seven factors interact to generate PFMreform outputs, which in turn should permit the achievement of PFM reformoutcomes. These factors are (i) the externally and (ii) domestically financedinputs to PFM reform and five contextual factors, namely (iii) complementaryinputs by Development Partners, (iv) demand-side pressures for PFM reform,(v) political leadership and commitment to reform, (vi) financing space forreforms and (vii) policy space for reforms.The results emerging from our country studies and case histories nowallow us to make some judgements on the relative significance of each of thesefactors: he overall level of financing for PFM reform inputs was onlyT loosely correlated with achievementsin terms of improved out-comes. Each of these countries saw broadly similar levels of investment inPFM reforms, yet outcomes were considerably better in Burkina Faso thanin the other two countries. overnment funding for PFM reforms was substantial and fre-G quently substituted for donor funding.In each country, we estimatethat Government funding amounted to 30 –50 % of total funding for PFMreform efforts over the ten-year period. Moreover, when donor funding forreforms, which did enjoy political support, was stopped or stalled, nationalbudget funding was often procured. The outstanding example is the US$14 million spent by the Malawi Government on its IFMIS over 2004 to2009, but each of the three countries exhibited high levels of governmentfunding for PFM reform (See Table 5, above.) he level of Donor funding for PFM reforms seems to haveT been more poorly correlated with outcomes than Governmentfinancing.The quality of the data is not sufficient to generate a precisemapping from funding sources, to reform outputs and reform outcomesand, in each of the three countries, reform initiatives were often jointlyfinanced by Government and its external partners. Nevertheless, Govern-ment funding does appear to have been allocated to genuine reform priori-ties. By contrast, a significant proportion of donor funding was directedtowards technical assistance activities, which were outside of the Govern-ment’s PFM reform priorities and for which there was no political supportand commitment. These activities resulted either in reports, which gener-ated no real output or in outputs, which never achieved full functionality37. he space for domestic funding of PFM reforms was consider-T ably expanded by the inflows of General Budget Support (GBS).In Burkina Faso and Malawi especially, but to a degree also in Ghana,37 We discuss in Chapter 4, section 4.3 the reasons for the continued inability of themanagement mechanisms to prevent technical assistance initiatives outside of theagreed PFM reform programme from proceeding.

GBS provided much higher levels of discretionary budget funding38thanthere would otherwise have been. s aconsequence, limitations in “financing space” were gener-A ally not aconstraint to the implementation of reforms.This wasboth because donor funding was generally abundant, and because govern-ment funding was available to substitute for external funding where neces-sary. The one partial exception to this finding is Malawi during the period2004–2009, when higher levels of external financing could potentially havebeen useful, given the favourable reform climate of this period. omplementary actions by donors to promote increased use ofC country systems were generally not effective.In several cases,problems arose because country systems were partially, rather than fullyadopted. Thus, a number of common basket fund or trust fund arrange-ments were created to earmark funding to certain areas of public spending,such as the HIPC trust fund in Burkina Faso, targeted to “poverty reduc-ing” spending, and “SWAp arrangements” for education, health or waterin each of the three countries. In the country studies, most of these fundswere reported to have created difficulties in disbursement, budget execu-tion and financial reporting. Disbursements of Budget Support were alsofrequently later than projected, notably in Burkina Faso and Malawi, cre-ating treasury management problems for government. tructures created for PFM reform monitoring and dialogueS under Budget Support were effectivein accelerating the preparationof comprehensive PFM reform plans, in facilitating the monitoring of pro-gress and in promoting harmonised external support to PFM reform. Bycontrast, efforts to use Budget Support conditionality to promote fasterprogress in PFM reform or the adoption of new reform measures were notsuccessful and, in some cases, undermined the effectiveness of the monitor-ing arrangements39. ivil Society Organisations were not asignificant influenceC over the content or the pace of PFM reform.Although the countrystudies did not explicitly examine the influence on the quality of PFM ofnational professional associations (e.g. of accountants or auditors) or of user38 By “discretionary”, we refer to domestic funding over which the Government has gen-uine allocative choice. A high proportion of budget funding in Developing Countriesneeds to be assigned to expenditures over which there is little or no discretion, suchas debt servicing, the funding of constitutional office holders, pension obligations,public sector salaries and essential operating & maintenance costs. One of the mostsignificant contributions of Budget Support is to increase the space for discretionaryspending.39 The evaluation of Ghana’s Multi-Donor Budget Support arrangement (Lawson etal, 2007) noted that in 2005 and 2006, the Government of Ghana claimed muchfaster levels of progress in the roll-out of the BPEMS system than were actually be-ing achieved, probably influenced by the fact that this was a “trigger condition” forBudget Support. The truth emerged in 2007, when the MDBS donors agreed to waivethe BPEMS trigger as ‘no longer relevant’. The evaluation team considered that theuse of conditionality had inhibited full and frank discussion of the problems at anearlier stage.

groups (e.g., for public education or health services), the country studiesfound the influence of CSOs as a whole to be minimal, even though theydid participate directly in Burkina Faso’s steering group for PFM reform(monitoring the SRFP), and in related Budget Support and PRSP reviewprocesses in all three countries. Perhaps the most powerful illustration oftheir limited influence is the fact that in the most recent PEFA assessmentsnone of the study countries scored higher than a “C” on public access to fis-cal information. he role of the Legislature in promoting PFM reform was alsoT generally limited.In Burkina Faso, all stakeholders interviewed agreedthat the Legislature’s influence on the pace and content of PFM reform hadbeen wholly insignificant. In Ghana, the public hearings of the PublicAccounts Committee were reported to have been influential in increasingpressure for continued improvements to external audit, but no influence onother aspects of the PFM reform programme was reported. In Malawi, itwas considered that the pressure of the Legislature had helped to focusattention on PFM reform over the 2004–2009 periods, but it was nota decisive influence and did not persist after the 2009 elections, when thePresident’s DPP party won a parliamentary majority. egional bodies did significantly influence the content of PFMR reforms, most notably the WAEMU Commission.In BurkinaFaso, the decision to establish an independentCour des Comptes(Court ofAccounts) in 2000 was reported to have been influenced by the WAEMU’sadoption of a directive requiring an independent audit function. The morerecent moves to deconcentrate authority for the authorisation of expendi-ture and to introduce a programmatic classification were also attributed tothe influence of the new WAEMU directives on these issues. In Ghana andMalawi, regional professional associations, such as AFROSAI, and peer-to-peer experience sharing networks, such as CABRI, were reported tohave had a positive influence over the design of PFM reforms. he most significant contextual influence on the success ofT PFM reform was the nature and form of political support toPFM reform.However, political support and opposition to reform wasseen to come from different places for different types of reforms and even inGhana, where the lack of active support was identified as an impediment toreform, it proved possible to achieve considerable success in the reform ofrevenue administration. Thus, it is essential to understand exactly how andwhy the nature of political support can be influential – an issue we examinein detail below. he other key constraint to successful PFM reform lay in theT limited policy space available to countries in designing andimplementing PFM reform interventions.By “policy space”, werefer to the depth, breadth and suitability of the menu of ideas, whichshape the design and implementation of PFM reform: in all three coun-tries, specific policy directions were pursued over long periods, eventhough, when viewed retrospectively, they were obviously wrong or inap-propriate. This was particularly the case with regard to reform policies on

3.3 Why And hOW iS POliTiCAlCOMMiTMEnT TO PFM REFORMiMPORTAnT?It is an accepted truth that technical ownership and leadership of reforms isnecessary if they are to succeed. Our case studies suggest thatin relation toPFM reform, technical ownership and leadership is anecessarybut not sufficient condition for success.It needs to be backed up bya political commitment to those reforms. Why is this? And what form doesthat political commitment need to take? These questions must be addressed ifwe are to formulate a more refined hypothesis regarding political leadershipof PFM reform – informed by our empirical case studies.It is instructive to start with a comparison of the professional backgroundsand career paths of the Ministers of Finance in each of the three countries.Tables 9, 10 and 11 in Annex One present this data in detail. Clear patternsemerge in the selection of Ministers of Finance in the three countries: n terms of professional backgrounds, the Ministers of Finance who haveI served in these countries over the past 15 years can be said to have appro-priate and generally strong professional training. Most have been account-ants, public administration specialists, economists or bankers. The oneobvious exception is Malawi’s Friday Jumbe (Minister of Finance,2000 –2004), who was brought into Government by President Muluzibecause he was a senior Party member of UDF, rather than because of histechnical background. urkina Faso’s Ministers of Finance have all been brought up through theB ranks of the Ministry of Finance or the Central Bank. One was a formerGovernor of the Central Bank before becoming Minister of Finance, themost recent three Ministers of Finance were all previously Deputy Minis-ters of Finance and two of them had held senior civil service positions inthe Finance Ministry. Thus, the current Minister of Finance, Lucien Bem-bamba, has been in senior positions within the Ministry since 1993. n Ghana and Malawi, by contrast, Ministers of Finance have generally beenI selected from Party ranks, from the banking sector or from academia. Theywould thus all have arrived into the post with less knowledge of the specificsof the PFM system in their countries than their counterparts in Burkina Faso.

tions, after finishing at the Ministry of Finance. Two of them went on to bePrime Ministers and one became Deputy Governor and then interim Gov-ernor at the BCEAO. Thus, from the day of his appointment, Lucien Bem-bamba, the current Minister, enjoyed the support of one ex-Minister ofFinance as Prime Minister, and another at the Central Bank. his degree of continuity is not present in Ghana. In part, this is due to theT alternation of power between the NDC and NPP parties but even when thesame party has remained in power, ex-Ministers of Finance have notremained in Government. Dr Kwesi Botchwey, after his retirement in 1995took up a series of international positions, while Hon Yaw Osafo Maafowas only briefly Minister of Education in the second Kuffour Governmentbefore returning to the private sector. alawi has shown more continuity than Ghana, most particularly duringM 2004–2009, the period when most progress was made with PFM reforms.Dr. Cassim Chilumpha, who had been Finance Minister over 1998–2000was brought into Government as Vice-President, while Dr. GoodwillGondwe, an economist who had held senior positions in the IMF, wasmade Minister of Finance. In a sign of the change of direction after the2009 elections, Dr. Chilumpha left Government and Dr. Gondwe waseffectively demoted to the Ministry of Local Government, before leavingGovernment and being brought back as Minister of Energy in 2011.These differences go a long way to explain the different levels of political com-mitment to PFM reform, which we have observed across the three countries.Political commitment to reform has been at the heart of Burkina Faso’s successin implementing its PFM reforms. This political commitment has been:IConsistent and long-standing– the current Minister of Finance hashad a senior role in PFM reforms since 1993, as have many of his staffwithin the ministry;IIDeep– the Minister of Finance enjoys support for the PFM reform agen-da from the President but also from the Prime Minister and the InterimBCEAO governor; he can also rely on a strong team of technicians withinthe SP-PPF, the secretariat coordinating the reforms, and amongst theDirectors within MoF and other ministries actually implementingreforms, a number of whom would have been his former civil service col-leagues; andIIITechnically informed– for most of the reform period, the people occu-pying the posts of Minister of Finance and Prime Minister have been PFMspecialists. There has also been an active process of engagement of otherMinisters in the PFM reform, through the approval by the Council of Min-isters of the current and the previous PFM reform plans (PRGB & SRFP).At an operational level, in what way has this type of political commitmentmade a difference to how efficiently and effectively PFM reform has beenimplemented? Clearly, it would have facilitated approval of national budget-

ary funding, whenever this has been necessary. It would also have helped toprevent, or at least minimise, the hindrances created to PFM reform by inter-departmental “turf disputes”. Yet, perhaps the most important operationalconsequence has been that institutional heads and Directors General in MoFand elsewhere have either been deliberately selected because of their ability tolead reforms or have quickly understood that future career developmentwould be influenced by their ability to learn how to deliver on the reformagenda. In short, in Burkina Faso, political commitment has served to createa commitment to reform amongst managers and technicians.By contrast, in Ghana, successive Accountants General were content toallow a major reform of financial management (BPEMS) to be planned andmanaged by consultants, without intervening to ensure that the design wasappropriate and to correct implementation problems. If their political mastershad been genuinely committed to the BPEMS reform, it seems quite incon-ceivable that this would have been allowed to happen. Fyson (2009) expressedthis as follows:‘It is not surprising that consultants should have been at the forefront of the knowledge trans-ferred but what was astonishing was the total lack of government participation in definingand shaping that knowledge to Ghana’s needs.’In Malawi, Bakili Muluzi’s second term as President over 1999 to 2004was the low point of the PFM reform programme. This term saw a progressivedecline in the quality of fiscal management, increasing corruption and gener-ally declining standards of public management (Barnett, Chisvo et al., 2006).It was characterised by macro-economic instability caused by unsustainablefiscal policies but also by institutional weakening with political loyalists (suchas Friday Jumbe) appointed to head ministries and run key institutions such asthe electoral commission (Booth, Cammack et al, 2005, p. 25). Corruptionbecame widespread with many civil servants copying the behaviour of seniorofficials and politicians, demanding fees for public services for private gain.Thus, the political leadership set the tone for how civil servants could behave,making it difficult for a commitment to reform to be protected within the civilservice.Clearly, the combination of positive factors listed above has been difficultto bring together within the more competitive political environments of Mala-wi and Ghana. It would require a more independent civil service, so thatgreater continuity could be maintained amongst senior officials within theMinistry of Finance and elsewhere, despite changes of government. It wouldalso need a greater level of cross-party agreement on public finance manage-ment and reform, so that consistent support for the most important PFMreforms could be obtained. However, these changes are difficult to achieve incontexts where party politics must necessarily occupy much of the time andattention of the President and the senior Ministers.Over the past decade, the most successful cases of PFM reform in Ghanaand Malawi have emerged when it has been possible to overcome these inher-ent obstacles:70

 hana’s revenue reforms were pursued consistently from 1995 onwards,G enjoying senior political support from both parties, which in turn allowedfor consistency in managerial and technical leadership. alawi’s IFMIS was introduced in a period of 18 months and rolled outM over 26 months, after a supportive Vice-President and Minister of Financewere appointed, a new Accountant General brought in, and US$ 14 mil-lion of domestic budgetary resources were allocated to the task.What lessons do we therefore draw for our Programme theory? Our mainconclusion is that political commitment to PFM reform is not simply an “add-ed bonus”. Nor is it true that political acquiescence in reforms is sufficient forthem to proceed and be sustained, so long as there is good technical leader-ship. Only for the most narrow, technically driven reforms might this be truebecause without political commitment, good technical leadership is unlikelyto be sustained over the course of reforms. Moreover, technical leadership willhave neither the political power to overcome resistance when it emerges northe influence to command additional budgetary resources when they areneeded.Thus, our case studies suggest that political leadership is a necessary con-dition for achieving sustained success with PFM reforms. This is a significant-ly stronger conclusion than that reached in the literature quoted in Pretorius& Pretorius (2008) and in De Renzio (2009), which identifies political leader-ship more as a contributory factor than a necessary condition. However, Rob-inson (2007) does identify the ‘nature of political agency’ as a critical successfactor in governance reforms in Brazil, India and Uganda. Moreover, in twobooks not included in the literature reviews, Graham Scott and Kate Jen-kins – leading public servants who guided PFM and public sector reform inNew Zealand and the UK – attest to the necessity of political commitment toreform40. It is also telling that the African government officials involved in theTunis Inception workshop for this evaluation emphasised the essential impor-tance of political leadership to the success of PFM reforms. (See Chapter 1).Further case study work will be needed to refine the understanding of thetype of political leadership, which is necessary for successful PFM reform.What is clear is that consistent, long-term, deep and technicallyinformed commitment to PFM reform is more likely to getresults, more especially when the PFM reforms being contem-40 Scott, G. (2001), Public Sector Management in New Zealand: Lessons & Challenges,Wellington, and Jenkins, K. (2006), Politicians & Public Services: ImplementingChange in a Clash of Cultures, London: Edward Elgar Publishing. Graham Scott wasSecretary to the New Zealand Treasury over 1986 to 1993 and is generally acknowl-edged as the technical architect of New Zealand’s PFM reforms. His book emphasizesthe importance of the political support provided by Prime Minister David Lange andhis immediate successors, under the pressure of an acute fiscal crisis and notes thechange of direction in reform introduced after 1999 by Prime Minister Helen Clark.Kate Jenkins was Deputy Head of the Prime Minister’s Efficiency Unit under the UKPrime Ministers, Margaret Thatcher and John Major. She highlights the need forpolitical leadership to overcome civil service opposition to public sector reform.

3.4 undERSTAndinG “POliCy SPACE”COnSTRAinTSBy “policy space”, we refer to the depth, breadth and suitability of the menu ofideas, which shape the design and implementation of PFM reform. Thephrase involves a rather particular usage of the word “policy”, but the ideabehind the phrase is that PFM reform policy space is essentially that range ofideas, which is open for consideration by governments wanting to initiate ordeepen PFM reforms.In many developing countries, the degree of familiarity of governmentofficials and politicians with PFM reform models is relatively limited, andmany such countries thus look to the Development Agencies and their consult-ants for guidance over the types of reform models, which they should consider.In each of the three case study countries this was the case. All countries gener-ally valued the external advice they received – even in retrospect. Neverthe-less, in all three countries we identified certain PFM reform ideas and direc-tions, which continued to be pursued over long periods, even though, whenviewed retrospectively, they were obviously wrong or inappropriate.Clearly, political and institutional factors do play a role in the persistentapplication of inappropriate reform models: for example, the project coordi-nators, within government and within the concerned Development Agency,might have their reputations seriously tarnished by admitting to mistakes; theconsultants implementing the project might stand to lose financially, and therewould be administrative and financial costs to government from the lostinvestments. Thus, one can understand why fundamental changes in thedesign of reforms might be slow to be introduced.However, common to all of the examples of policy space constraints ineach country was the fact that, at the outset, government authorities were pre-sented with a limited range of options by their Development Partners andwere not encouraged either to consult or to reflect more widely on the choiceof reform models. There were apparently neither financial constraints norpolitical constraints determining the choice of reform model, so we interpretthis as simply as a lack of adequate ideas: a policy space constraint.Five of the nine case histories analysed threw up examples of policy spaceconstraints: n Malawi, Procurement reforms and Internal Audit reforms were both Ipremised on the use of ‘best practise’ models in which responsibilitieswould be decentralised. This would have required new skills to be devel-

oped and embedded in virtually all government institutions. These modelswere fundamentally inappropriate to a context of significant shortages oftrained staff within the civil service, combined with the persistence of hier-archical modes of working, limited accountability and a culture of frequentdisregard of rules. Simpler reform models were needed, less demanding oforganisational and human resource capacities. he BPEMS reform in Ghana was based on a technology-driven model,T which gave insufficient attention to change management issues, and to theassessment of capacity constraints and training needs. It was also based onan ambitious “big bang reform” approach rather than a more gradual,“incrementalist” approach. Again this was a ‘best practise’ model, whichwas inappropriate to the context. he MTEF reform in Ghana presented a different type of policy spaceT constraint, in that the consultants leading its introduction selected and pro-moted an idiosyncratic model, which was never likely to achieve thedesired outcomes of an MTEF. The reform design was idiosyncratic firstly,in placing emphasis on the “bottom-up” elements of an MTEF (pro-gramme-based budgeting, detailed costing of plans, integration of aid-financed projects) and barely mentioning the upstream elements (macro-fiscal framework, strategic policy and expenditure review processes) thatare conventionally seen as the initial priorities in establishing a robustMTEF process. Secondly, it adopted an activity-based budgeting approachto the development of multi-year budgets, which generated voluminous,highly detailed documents, which made it very difficult to discern the strat-egies and priorities underlying MDA budgets, thus effectively defeating thepurpose of the exercise. n Burkina Faso, the problem was essentially a failure to sequence work onI the medium term expenditure framework and programme budgets in anadequate manner. It involved the attempted introduction of programmebudgets before there was an aggregate medium term fiscal framework inplace, when many sector policy and strategy frameworks were unclear andwhen the processes for tracking performance during budget execution atthe programme level were yet to be designed.Why was there not a more appropriate set of reform models presented to thegovernments of Burkina Faso, Ghana and Malawi? What caused these “policyspace” constraints? We believe there were four main factors at play:IInadequate knowledge and experience of PFM reform,on thepart of Development Agencies and their consultants clearly played a part.In this respect, it is important to recall that even now in 2011, PFM reformis a relatively under-researched area: the world is still learning what typesof reform work best in different contexts. Several of the examples of policyspace constraints date back to the late 1990s, when knowledge of the sub-ject was in many ways in its infancy.IIA tendency to seek best practise modelsand apply them indiscrim-inately without concern for contextual differences was also an aspect of theproblem. This tendency, which has been labelled “isomorphism”

(Andrews, 2009) was not just prevalent within the Development Agencies:politicians and senior government officials in the case study countries hadin several cases seen what they interpreted as best practise and wanted itapplied directly in their own countries. Thus, the Minister of Finance ofBurkina Faso had seen the Canadian approach to programme budgetingand wanted the same model to be applied. Ghanaian officials had under-taken study tours to Australia and New Zealand and wanted the sametype of integrated financial management systems to be introduced in Gha-na. They were deliberately seeking to import ‘best practise’ and gave con-sultants carte blanche to do this41.IIIWeaknesses in the Development Agencies’ supervision andpeer review processes for PFM reform projectswere probablya relevant factor. The evaluation team has not had the time or resources toexamine supervision processes, so this observation is essentially a matterof conjecture. However, PFM reforms are significantly different to thestandard projects, which provide the frame of reference for the design ofsupervision and peer review processes within Development Agencies. Inparticular, most projects can be developed to a high level of detail at thedesign stage and hence many quality assurance processes in DevelopmentAgencies are “front-loaded” to allow for a detailed appraisal of projectdesign. PFM reform projects, by contrast, cannot normally be defined inextensive detail at the design stage because they usually rely on detaileddiagnostic work being undertaken during the inception phase. They alsotend to require adaptive designs, which evolve as implementation pro-ceeds. Thus, peer review for PFM reforms is likely to be more useful atinception stage and would generally need to be more continuous than forother projects. We would judge that few, if any, Development Agencies areeffectively geared up for this type of support.IVGovernments’ own monitoring and review processes werealso too weakto identify design mistakes and promote their prompt cor-rection. This was especially true of Ghana, and to a slightly lesser extent ofMalawi, each of which had significant weaknesses in the structures theyestablished for the coordination and monitoring of their PFM reforms.Burkina Faso’s systems were considerably stronger, which allowed fora more effective learning process during implementation – even if, in thecase of programme budgeting/ MTEFs the learning process was slow. Wediscuss these issues more fully under “mechanisms” in chapter 4.There are some signs that capacities for reform design and supervision withinGovernments and Development Agencies have developed since the earlieryears of our evaluation period. The CABRI network did not exist at that time;41 Fyson (2009) describes the resulting process as follows: ‘donor-funded consultants, asthe perceived legitimate holders of best practices in the implementation of governmentfinancial management systems, were given free reign in the determination of how tostrengthen accountability in government finances, often to the detriment of govern-ment participation and understanding.’

there were many less training courses and seminars on PFM reform availablein those years and there was less research available on PFM reform. Withinthe World Bank, Westcott (2008) records that the numbers of PFM specialistshas risen since 2000 and also identifies the positive influence of the FinancialManagement Sector Boards and of the Public Finance Thematic Group onthe consistency of PFM support operations. Thus, it may be that “policyspace” constraints have been eased over time.Nevertheless, the three case study countries are broadly representative ofthe range of experiences of PFM reform, which are likely to be found in Afri-ca. The fact that the success of PFM reform in these three countries was sofrequently undermined by the selection of inappropriate reform models shouldnot be taken merely as a coincidence. Policy space constraints have historical-ly been of considerable influence for the outcomes of PFM reform: it seemsprudent to assume within our programme theory that they are likely to con-tinue to be important.

3.5 AddiTiOnAl COnTExTuAl FACTORSFOR FuRThER invESTiGATiOnAn evaluation framework is necessarily a simplification of reality. It is pre-pared with the explicit intention of focussing attention on those factors mostlikely to be critical in the production of certain types of outcomes. In the pro-cess of applying any evaluation framework, it is therefore important to beaware of new factors, not fully captured within the framework, which emergeduring fieldwork. Two such factors have come to light. They concern, firstly,the relative competence of the civil service in the study countries and, second-ly, the role of CSOs in promoting PFM reform.Regarding the issue of civil service competence,the quantitativestudy (De Renzio et al., 2010) found that economic factors are most importantin explaining differences in PFM outcomes. Specifically, it found that ‘coun-tries with higher levels of per capita income, with larger populations and witha better recent economic growth record are characterised by better qualityPFM systems. On the other hand, state fragility,.….has a negative effect onthe quality of PFM systems.’ The competence of the civil service is positivelycorrelated with per capita income and negatively with state fragility, so wemay take this finding as providing support for the view that the humanresource and organisational capabilities of the public administration systemare strong determinants of PFM reform outcomes.The wider literature on the determinants of successful governance andpublic sector reforms also supports this view42. For example, Robinson (2007)in a detailed case study analysis of governance reforms in Brazil, India andUganda identifies as a key factor for success, ‘the degree of technical capacityand the level of insulation from external pressures of the bureaucracy.’

Therefore, a broader international comparison of the determinants of suc-cessful PFM reform should ideally incorporate within the analysis the effectsof differences in the relative competence of the public sector. How exactly thisought to be measured is not so clear, however. “Technical capacity” is deter-mined in part by the quality of staff entering the civil service, in part by thequality of training within the civil service, in part by managerial and organi-sational factors. It would be difficult to capture all of these factors withoutarriving at a tautological definition in which technical competence would beequated with quality of outcomes. It is precisely this methodological difficultythat frequently leads researchers to use per capita income as a simple proxy fordifferences in technical capacity. Is this a reasonable assumption? How rele-vant are those aspects of ‘technical capacity’, which do not correlate with dif-ferences in per capita income?In relation to our case study countries, an implicit assumption has beenmade that, given the broad similarities in income levels and the fact that noneof the three countries is a “fragile state” (i.e. a conflict or post-conflict coun-try), the differences in the capabilities of the respective civil services are not sosignificant as to be critical to the outcome of PFM reforms. With the per capi-ta income of Ghana being nearly 40 per cent higher than that of Burkina Fasoand virtually double that of Malawi (see Table 3), this is certainly a braveassumption but, if anything, it would introduce bias in favour of Ghana,which, as we have seen, achieved PFM reform outcomes that compare poorlywith those of Burkina Faso43.Are there other aspects of technical capacity, which compensate forincome-related differences and which may help to explain the relatively strongperformance of Burkina Faso? Commentators on the draft synthesis reportdrew attention to two factors, which were considered important in differenti-ating the public administration of Burkina Faso from those of Ghana andMalawi: a) the quality of training received by financial managers; and b) therelatively high level of civil service salaries in Burkina Faso and the relativelylimited range of alternative opportunities within the private sector.The Burkina Faso country study did identify the quality of training infinancial management through the ENAREF (Ecole nationale des regiesfinancières: National Financial School) as a relevant contextual factor.ENAREF has a strong reputation in Francophone Africa and, indeed, attractsstudents from neighbouring francophone countries. It has probably playeda positive role in ensuring a high level of technical competence amongst thosecivil servants responsible for implementing PFM reforms. On the other hand,GIMPA, the Ghana Institute of Management & Public Administration alsoenjoys a good regional reputation, as do the Malawi Institute of Managementand the Malawi Staff Development Institute. Similarly, while there is nodirect comparison in the Anglophone tradition to the Francophone system offinancial management training, training in public sector accounting and43 In this respect, it is also noteworthy that net secondary school enrolment is 47 % inGhana, compared with 28 % in Malawi and only 16 % in Burkina Faso (World Devel-opment Indicators, 2010).

book-keeping is also highly structured within the Anglophone tradition, withinternationally benchmarked qualification systems. Whilst more detailedanalysis may be merited, on first examination this does not appear to bea strong differentiating factor.The issue of the relative competitiveness of civil service salaries does seemat first sight more relevant. It is evident that in the absence of the more vibrantprivate sector economy of Ghana, the public sector of Burkina Faso faces morelimited competition in attracting qualified staff. It is noteworthy that the prin-cipal area of progress in PFM reform in Ghana, namely revenue administra-tion, occurred within the Ghana Revenue Authority, which is an executiveagency, enjoying significantly higher salaries than the remainder of the civilservice.On the other hand, it is difficult to draw strong conclusions without a morecareful analysis: Malawi also has a small private sector and it is not clear thatpublic sector salaries in Burkina Faso are significantly more competitive. Aswe have seen, significant progress was made with the IFMIS in Malawi over2004 to 2008, without the benefit of salary supplements or executive agencystructures. Even in Ghana, while the outputs resulting from the MTEF andBPEMS reforms proved ultimately to be inappropriate to the desired out-comes, outputs were generated, involving the participation of large numbersof civil servants. The shortcomings of these reforms were more to do withinappropriate designs, which were not corrected as a consequence of poormechanisms of management and monitoring and a lack of political leadershipand engagement.In conclusion,we would recommend that future case studies doaim to incorporate more explicitly the issue of the relative com-petence of the civil service,which was not formally captured within theevaluation framework we have used. Nevertheless,we do not believe thatthere is sufficient evidence to suggest this was aprimary factorin explaining the differences identified in PFM reform outcomesacross our case study countries or our case histories.The limited influence of Civil Society Organisations on thedesign and pace of reformswas a surprise to several commentators.A more extensive and more multi-faceted analysis of this issue would perhapshave revealed a range of more subtle influences, than the simple “pressure onthe Executive to reform” which the country cases sought. For example, ifregional bodies and peer-to-peer learning organisations can be influential, it iscertainly conceivable that national professional associations would have a sim-ilar type of influence on reform agendas.It may be that the lack of influence derives partly from the limited exper-tise of the CSOs working on budgeting and public spending, and/or from thelimited engagement in advocacy work of the relevant professional associations(such as the Institute of Chartered Accountants of Ghana, the Society ofAccountants in Malawi and theAssociation de Compatbilité Nationaleof BurkinaFaso.) These were relevant factors in the case study countries.However, one should recall that in the case study countries, there isa range of societal factors that mitigate against the ability of normal citizens

and their associations to hold the government to account, through the mecha-nisms of advocacy, media discussion and protest. The Afrobarometer surveyhas been conducted periodically in these and other African countries a num-ber of times over the past decade. Some of the most interesting questions inthe survey relate to the attitudes of citizens towards their governments. Onequestion asks respondents to choose which of two phrases better reflects theirview of government. The table below shows the answers recorded in the casestudy countries, when the survey was most recently run, over 2008 and 2009.In each country, the overwhelming majority of respondents looked to theirgovernments to “take care of them” and only the minority felt that they shouldbe “the bosses, who control the government”. Such attitudes do not createa strong environment for citizens to challenge their governments to be moreefficient.

In summary, we would judge thatgiven the relatively passive attitudesof citizens towards their Governments in the case study coun-tries, it is not surprising that advocacy work by CSOs in supportof stronger PFM systems has not had astrong influence on PFMreforms within the time-frame under consideration.This does notmean that CSO work on such issues has been fruitless. It may help over thelonger term to create greater public awareness on PFM issues and thus tostrengthen attitudes towards transparency and public accountability. But itseems safe to say that this must be a long-term objective: CSO pressure toimprove PFM will not significantly affect the outcomes of PFM reforms in theshort to medium term. More targeted campaigning on specific issues – such aspublic access to fiscal information – may help to obtain faster results but thiswas not a feature of CSO work in the case study countries.

4.2 ThE MOST iMPORTAnT ASPECTS OF ThEMAnAGEMEnT MEChAniSMS FOR PFMREFORMThe country studies and case histories reviewed in chapter 2 illustrate verypowerfully the importance of strong management arrangements for PFMreform. Indeed, this factor – combined with the strong political commitmentto PFM reform serves to explain much of the difference in performance seenbetween Burkina Faso and the other two countries. What exactly were the fea-tures of Burkina Faso’s arrangements for the management and coordination ofPFM reform, which made them effective?At the beginning of the evaluation period, Burkina Faso saw a shift froma PFM reform programme, based on a range of loosely coordinated individu-al initiatives to the development of an integrated programme of PFM reforms,developed on a consultative basis and endorsed at the highest political level. InJuly 2002, the Council of Ministers adopted government’s programme tostrengthen budget management (Plan d’Actions pour le Renforcement de laGestion Budgétaire – PRGB). The PGRB incorporated recommendations ofseveral earlier assessments, including the 2000 Country Procurement Assess-ment Report (CPAR), the 2001 Country Financial Accountability Assessment(CFAA), the 2001 HIPC AAP (2001), and the IMF’s Report on the Obser-vance of Standards and Codes (ROSC) of 2002.TheSecrétariat permanent pour le suivi des politiques et programmes financiers(SP-PPF) was also created in 2002 to take charge of coordination of PFM reformsand monitoring of budget support programmes. The SP-PPF’s predecessorwas the Technical Secretariat for the Coordination of Economic and SocialDevelopment Programmes (STC-PDES), which had had a similar set of func-tions. Both the STC-PDES and the SP-PPF were and remain powerful bodies,reporting directly to the Minister of Finance, and having the status to attract

high calibre personnel. The SP-PPF’s coordinating role was strengthened bythe introduction of the PRGB in 2002 and the development of a coordinatedframework for the management of Budget Support, with the signing in 2002of the MoU for theSoutien Budgétaire Commun auCSLP (SBC-CSLP).Based on the lessons learnt from the PRGB implementation, the BurkinaFaso Council of Ministers adopted a Strategy to Strengthen Public Finance(SRFP) in April 2007. The SRFP has a time-line of nine years (until 2015),and is accompanied by a rolling three year action plan (Plan d’Action Secto-riel Triennal – PAST).The implementation of both the PRGB and the SRFP have been overseenby a Steering Committee chaired by the Minister of Economy and Financecomprising: (i) representatives of the ministries and institutions which areinvolved in implementing the SRFP, (ii) the Development Partners (DPs) pro-viding financial support to the SRFP, and (iii) representatives of Civil SocietyOrganisations (CSOs). The Steering Committee meets twice a year. The SP-PPF comprises the technical secretariat of the Steering Committee and hasresponsibility for monthly monitoring of progress with the PRGB/ SRFP. Inaddition, six technical groups have been set up under the responsibility of theSP-PPF: budget management, resource mobilisation, public procurement,internal control, deconcentration and decentralisation, and capacity strength-ening. These six technical groups bring together the institutions responsiblefor the different components of the PRGB/ SRFP, as well as civil society anddonor representatives. They report directly to the SP-PPF but, in addition, arechaired directly by the conseillers techniques (technical advisors) of the Minis-ter of Economy and Finance, creating a direct link between the operationaland political levels.Overall, the management arrangements for PFM reforms in Burkina Fasohave been impressive, embodying a number of positive features: n integrated PFM reform programme, developed through a consultativeA process drawing on diagnostic assessments (PEFA, CFAA, CPAR, ROSC),and endorsed at Cabinet level. management and monitoring team of high calibre local staff, of highA authority and with direct links to the Minister of Finance (the SP-PPF). n implementation framework led by the institutions and agencies withA statutory responsibility for the functions being reformed (such as therespective Directions Générales of MEF), working under the close coordi-nation of the SP-PPF. clear and respected mechanism of monitoring and evaluation, incorpo-A rating periodic PEFA evaluations. harmonised framework for the provision of donor support to the com-A mon PFM reform programme and for regular dialogue on PFM reformissues.Undoubtedly the most important of these features has been thecombination of clear implementation responsibilities for the rel-evant Directors and unit Heads, with authoritative oversight andcoordination by acentral team reporting to the Minister of

Finance.In Ghana and Malawi, significant efforts were dedicated to theconstruction of management and coordination mechanisms. These effortsbrought some success – in terms of the development of integrated pro-grammes, common monitoring mechanisms and harmonised dialogue struc-tures, but the most striking difference is that neither country succeeded in cre-ating this framework combining clear (and accepted) operational responsibili-ties for PFM reform, with authoritative monitoring, coordination and guid-ance from the centre.Malawi’s greatest PFM reform success – the introduction of the IFMISover 2004 to 2008 – was largely due to the authority for implementation vest-ed in the Accountant General but the counterbalancing central function ofcoordination and monitoring, managed through the PFEM (Public Finance &Economic Management) Technical and Steering Committees never operatedas effectively as the SP-PPF in Burkina Faso. It was not capable of pressing theprevious Accountant General (prior to 2004) to change direction when Mala-wi’s initial IFMIS design was failing; nor more recently, has it been able tostimulate the staff responsible for Internal Audit or Procurement to developnew approaches more suited to the prevailing capacity constraints than thecurrent decentralised, staff-intensive models.The BPEMS and MTEF reforms in Ghana suffered from shortcomingsboth in the definition of implementation responsibilities and in the establish-ment of an effective oversight and monitoring function. In large part, this wasdue to their heavy reliance on consultants44. BPEMS and MTEF were bothimplemented directly by teams of long-term consultants contracted for thispurpose. The Accountant General and the Budget Director received reportsfrom the implementing consultants but their sense of operational responsibil-ity for the reforms was distant, particularly with regard to the AccountantGeneral and BPEMS. The activities of the implementing consultants were inturn coordinated by the PUFMARP Project Implementation Unit, which wasalso made up of consultants. Although the PUFMARP PIU reported toa Steering Committee, made up of senior government officials coordinated bythe Deputy Minister of Finance, at this senior level responsibility for thereforms was also diffuse. As we have noted in Chapter 2, it did not have thevision and influence to engage the necessary departments in converting theMTEF from an activity-based budgeting exercise into a tool of medium termstrategic planning, nor did it have the sense of urgency to raise alarm bellsand change direction, when BPEMS was failing to progress.Notwithstanding the achievements of the case study countries, there aretwo aspects of the management and coordination mechanisms, which wereweak in all three countries, with negative effects both for the efficiency of thereform process and for the quality of final outcomes. We refer specifically toweaknesses in (i) the coordination of technical assistance support to PFM44 We have not been able to analyse in depth the use of consultants across the threecountries but in Burkina Faso, although there have been long term consultants in-volved in PFM reforms, these have always had an advisory rather than a managerialor implementation role. Moreover, over time, the number of long-term (resident) con-sultants has been steadily limited, as indeed has happened also in Malawi and Ghana.

4.3 ThE COnTinuinG WEAknESSES in ThECOORdinATiOn OF TEChniCAlASSiSTAnCEIn the three case study countries,asignificant proportion of donorfunding was directed towards technical assistance activities,which were outside of the Government’s PFM reform prioritiesand for which there was no political support and commitment.These activities resulted either in reports, which generated no real output or inoutputs, which never achieved full functionality: in short, in PFM reformactivities, which were both inefficient and ineffective.In general, the number of cases of uncoordinated (and inefficient) techni-cal assistance support to PFM reform appears to have declined over time butsuch cases were commonplace over much of the evaluation period. How canthis phenomenon be explained? Was it a consequence of the respective Gov-ernments having weak donor coordination mechanisms? Or did it derive frominappropriate policies on technical assistance by the respective Developmentinstitutions? Would the use of different modalities for provision of technicalassistance have improved efficiency and effectiveness? We examine each ofthese hypotheses in turn.In relation to coordination mechanisms, it should be stated that all threecountries exhibited good progress in the development of harmonised frame-works for the provision of donor support to PFM reform and for dialogue onPFM reform issues. At an early stage of the reform process, the three Govern-ments were successful in establishing harmonised frameworks for support toPFM reform, based around explicit programmes of PFM reforms, joint moni-toring processes, common dialogue frameworks and formalised agreements towork within the joint programmes (such as memoranda of understanding). Ineach case, the majority of Development Partners supporting PFM reformssigned up to these agreements and collaborated actively in the establishmentof these harmonised frameworks.There were weaknesses in these frameworks (as we note in Table 7, inChapter 3) – in Ghana, there was a three year gap between the close of thePUFMARP in 2003 and the launch of the Short & Medium Term ActionPlan on PFM (ST/MTAP); in Malawi, there were consistent weaknesses inthe PFEM Action Plan, which the Malawi country report assesses as ‘anamalgam of different reform interests, rather than a coordinated andsequenced response to PFM weaknesses’. (Folscher, et al., 2011). It is often saidof harmonisation frameworks that “the devil is in the detail”, and more atten-tion to the definition of PFM priorities and to the establishment of rules andnorms governing technical assistance support to the respective PFM reformprogrammes would probably have been useful.82

Nevertheless, it seems difficult to argue that stronger harmonisationframeworks would have significantly reduced the incidence of TA activitiesoutside of the Governments’ PFM reform programmes. In the first place, it isin Burkina Faso, where coordination frameworks were clearly stronger, thatwe find the most notable examples. These include the IMF TA missions onrevenue reform, which repeatedly presented designs for the phasing out of taxexonerations, which the Government was committed to protect, the SECOfunded support to tax reform which was channelled to activities outside of theGoBF reform plan (SRFP) and remained largely undisbursed, and the cross-cutting institutional support projects of the World Bank and the AfricanDevelopment Bank in the early stages of the reform programme, whose objec-tives were so loosely defined that outputs were never clear.These experiences suggest that the problem is not one of coun-try-level coordination mechanisms but an issue of the design andconduct of policies on technical assistance at the DP headquar-ters level.In particular, there is a need for the DPs’ TA activities in supportof PFM reform to be explicit about their objectives and their anticipated out-puts and outcomes, and secondly to be subjected to independent evaluation ona more systematic basis. It seems unlikely that the inefficient and ineffectiveTA activities conducted by Development Partners in support of PFM reformin Burkina Faso, Ghana and Malawi would have continued if systematic eval-uation had been taking place. In this respect, the international pressure formore systematic evaluation of development cooperation45is encouraging, as isthe generally favourable response of the majority of development agencies.46However, technical assistance cannot be effectively evaluated if its objec-tives are unclear and the anticipated outputs and outcomes are not clearlystated. The three country teams all encountered significant methodologicaldifficulties in categorising information on the inputs and outputs of TA activi-ties in support of PFM reform. This was due both to an information scarcityproblem – during the evaluation period, many TA projects in support of PFMreform simply did not produce progress reports – but also to a fundamentaldesign problem: many TA activities did not have their outputs defined ex anteand thus did not have a clear framework for reporting progress.Significant difficulties also arose in the case study countries due to the con-fusion of the different objectives of TA activities. Government representativescommented that it was often unclear which TA activities were supposed to bepart of the PFM reform programme and which were being done on behalf ofthe Development Agency. Whilst it is perfectly legitimate (and useful) to pur-sue different objectives with TA, an important step towards more effectivecoordination and harmonisation would be to distinguish more preciselybetween three types of TA activity:45 This has been led both by Legislatures demanding more value for money from devel-opment spending and by CSOs working in this area. A notable advocate is CGD, theCentre for Global Development, in Washington D.C.46 As an example of this, we understand that the IMF’s Independent Evaluation Groupis, at the time of writing, conducting an evaluation of the IMF’s technical assistanceon PFM.

 irect support to a partner government to assist in the design or implemen-d tation of its PFM reforms; support to CSOs, the Legislature (and possibly the Supreme Audit Institu-tion47) to improve the context for good public finance management; and dvisory or diagnostic work, undertaken directly for the Developmenta Partner either to identify fiduciary risks in the use of government systemsor to generate an independent assessment of PFM reform priorities.Where the intention is to support a Government’s PFM reform programme,then the activities undertaken must logically form a part of that programme,and therefore be subject to a set of common rules and norms on reporting,monitoring and evaluation. Burkina Faso, Ghana and Malawi were generallyable to make reasonable progress in the establishment of common monitoringframeworks but it was not always clear which TA activities were expected tobe conducted within that framework, and which outside of it.This is not to suggest that all technical assistance was misdirected. Over-all, TA was well appreciated by the recipient governments but more attentionto its precise objectives, outputs and outcomes and a greater sensitivity to therole of TA in relation to the Government PFM reform programme might haveraised significantly its efficiency and effectiveness.Would common basket funds for TA provision to the PFM reform pro-grammes have been helpful in improving its effectiveness? No such “TA fund”was operating in any of the case study countries during the evaluation period,so we have no empirical basis for answering this question. Such a fund hasbeen operating for a number of years in Mozambique and similar arrange-ments are in place in a number of Developing Countries. There is thereforesufficient experience internationally on which to reach some evaluative judge-ments, although to our knowledge no formal evaluations for TA funds forPFM reform have been conducted.Drawing on the wider experience of common basket funds for TA, createdfor different types of SWAps and Sector Programmes in Developing Coun-tries48, it seems that such funds often face a number of practical problemswhich tend to undermine their effectiveness: n the first place, not all Development Agencies are legally and constitu-I tionally capable of signing up to such arrangements. As a result, somepotentially good providers of TA (such as the IMF, with regard to PFM)are excluded. econdly, those Agencies that do have the legal capability to contribute to STA common basket funds often find it difficult to reach agreement over the47 The point here is that most Government PFM reform programmes logically focus onthe reforms led by the Executive. Sometimes work to strengthen scrutiny by the SAI orthe Legislature will also form part of the PFM reform programme and sometimes not.48 The experience with common basket funds and with SWAPs in general is presentedin Boesen, N. & D.Dietvorst (2007), SWAPS in Motion – Sector Wide Approaches:from an Aid delivery to a Sector development perspective, Copenhagen & Brussels:Train4Dev.net.

precise management arrangements. As a result, the transaction costsincurred in the set-up process are frequently high. hirdly, the use of joint arrangements often complicates the sourcing andT contracting of technical assistants, which creates additional delays andtransaction costs, while not always succeeding in sourcing better qualityTA than would be available through moread hoc,bi-lateral arrangements.The elements of TA common basket funds, which are more commonly foundto operate effectively and to be useful, are the joint identification of TA needs,the sharing of terms of reference and the use of joint reporting and monitoringframeworks. None of these elements in fact require a common basket fund inorder to be introduced as common working norms for support to a Govern-ment PFM reform programme. To a degree these elements were present ineach of the PFM reform coordination frameworks in the study countries butthey were not established as standard, agreed norms. Moreover, without well-defined objectives for DP-provided technical assistance, which distinguish thereal “client” for the TA and define outputs clearly, these sorts of norms wouldbe impossible to apply.Further strengthening of coordination mech-anisms for TA to PFM reform programmes is needed but so tooare changes in the policies and procedures for TA provision byDevelopment Partners.

4.4 ThE lACk OF An EFFECTivE lEARninGAnd AdAPTATiOn PROCESSIn Chapter 3, we identified the significant problems generated for the casestudy countries by the “policy space” constraints, which they faced in theimplementation of their PFM reforms. Essentially, we found that there werelimitations in the depth, breadth and suitability of the menu of ideas, whichshaped the design and implementation of PFM reform. As a result, significantefforts were dedicated to the pursuit of reform models, which were simply notappropriate to the respective institutional, organisational and human resourcecontexts.Although the initial adoption of an inappropriate reform model may beattributed to an external policy space constraint, the failure to change direc-tion subsequently suggests an important weakness in the management mecha-nisms for PFM reform: namely, the absence of an active learning and adapta-tion process. In all three countries there were PFM reform ideas and models,which continued to be pursued over long periods, even though, when viewedretrospectively, they were obviously inappropriate. How could managementmechanisms have been designed to avoid this?Learning and adaptation processes need to be introduced both into Gov-ernment mechanisms for coordination of PFM reforms and into the supervi-sion and peer review processes of the Development Agencies: overnments’ mechanisms for coordination of PFM reformG need to include arrangements both for monitoring of progress

and for evaluation of the adequacy of that progress.While moni-toring frameworks (of varying quality) were established in each of the studycountries, none had effective mechanisms for ongoing evaluation. By eval-uation, we mean a systematic process of assessing performance against cri-teria of efficiency, effectiveness, impact, relevance and sustainability. Inmost cases, such a process needs to be conducted by third parties who havenot been engaged in the design and management of reforms and can thusmaintain a degree of objectivity and independence. Common models forthe incorporation of a periodic evaluation function include: engagement of outside experts or groups of experts, who are con-thetracted to assess progress on an annual or two-yearly basis; creation of an internal team of “wise persons” or “eminent per-thesons”, of senior, generally retired civil servants and academics, whomight provide a periodic independent evaluation of performance49;and creation and pre-funding of evaluation funds, so that morethefocused evaluation exercises can be quickly launched, based on com-petitive tenders. evelopment Agencies’ supervision and peer review processes Dfor PFM reform projects need to be continuous rather thanperiodicas is commonly the case with more classic projects. PFM reformsare significantly different to the standard projects, which provide the frameof reference for the design of supervision and peer review processes withinDevelopment Agencies. Whereas most projects can be developed to a highlevel of detail at the design stage, PFM reform projects cannot normally bedefined in extensive detail at the design stage because they usually rely ondetailed diagnostic work undertaken during the inception phase. They alsotend to require adaptive designs, which evolve as implementation proceeds.Thus, peer review for PFM reforms is needed at inception stage (as well asfeasibility and design stage) and generally needs to be more continuousthan for other projects. The experience of the case study countries suggeststhat, at least in the recent past, Development Agencies have not been effec-tively geared up for this type of support.In addition to the formal inclusion of evaluation and external supervisionfunctions within the management process, the promotion of a learning andadaptation function also implies the organisation of more open “learningevents”. These learning events might, for example, take the form of conferenc-es, where international research on PFM reform might be presented and dis-cussed, or of more informal experience-sharing workshops, where experiencesof reform design and management might be shared between common stake-holders within a country or within a range of similar countries.49 The Peruvian Ministry of Finance has adopted a hybrid arrangement to ensure thisexternal evaluation and learning function. It has engaged an independent ‘Grupo deestudios’ (Study group) on PFM reform, which includes two Peruvian retired civil serv-ants and one international consultant.

5.1 kEy COnCluSiOnSWhere and why do PFM reforms deliver results?IPFM reforms deliver results when there is a strong political commitmentto their implementation, when reform designs and implementation mod-els are well tailored to the institutional and capacity context and whenstrong management and coordination arrangements – led by governmentofficials – are in place to monitor and guide reforms.II Strong leadership and commitment to reform is also needed at the techni-cal level. In the case study countries, this emerged naturally where therewas political commitment and leadership. By contrast, commitment atthe technical level was not sufficient to generate political commitment.III Neither external donor pressure nor domestic pressure from the Legisla-ture or Civil Society proved sufficient to generate political commitmentfor PFM reform, where it was lacking, although they may have contrib-uted to preserving it, where it already existed.IV A common weakness of the management and coordination mechanismsfor PFM reform was that they did not make adequate provision for theregular, independent evaluation of performance. As a result, the learningprocess essential to the continuous evolution and adaptation of reformdesigns and models was generally weak, with the result that correctiveprocesses were slow to kick in, where reforms were not proceeding well.V Direct funding for PFM reforms by governments is more substantial andmore common than generally supposed, particularly in contexts wherethe resources available for discretionary spending are boosted by thepresence of Budget Support. However, the case study countries frequentlyfound themselves facing a constraint in respect of the policy space forreforms, where the menu of available policy designs and models for PFMreform were not appropriate to the institutional and capacity context.Reform outcomes were more favourable where a better range of policyoptions was available or where the mechanisms for monitoring and coor-dination of reforms promoted lesson learning and adaptation during theimplementation process.87

Where and how does donor support to PFM reform efforts contrib-ute most effectively to results?VII Donor funding for PFM reform has facilitated its implementation in thosecountries where the context and mechanisms were right for success, andwhere external funding was focused on the Government’s reform pro-gramme. On the other hand, governments in the case study countriesshowed a willingness to fund PFM reforms directly and their ability to doso was significantly facilitated by the General Budget Support inflowsthey were receiving. Hence, in many cases, direct external funding forPFM reform may not be essential.VIII Donor pressure to develop comprehensive PFM reform plans and toestablish clearly defined monitoring frameworks has been a positive influ-ence in countries receiving Budget Support.IX By contrast, attempts to overtly influence either the pace or the content ofPFM reforms through Budget Support conditionality have been ineffec-tive and often counter-productive.X Donor promises to enhance the utilisation of country systems have notgenerally advanced very far. In the case study countries, the late disburse-ment of Budget Support and the partial use of country procedures havebeen inimical to good public finance management.XI On the other hand, when focused on the Government’s reform pro-gramme, external technical assistance and advisory support have been ofgreat help to PFM reform processes in the study countries and were gen-erally well appreciated by recipient governments.XII Nevertheless, the provision of poor advice and the promotion of inappro-priate reform models by external agencies remain an unfortunate featureof many PFM reform programmes. Greater attention to the appropriate-ness of reform models is needed, within an adaptive, learning approach toPFM reform implementation.

cal commitment to their implementation, when reform designs and imple-mentation models are tailored to the institutional and capacity context andwhen strong coordination arrangements – led by government officials – arein place to monitor and guide reforms. Where these essential conditionsare not in place, PFM reforms are unlikely to be successful. In such cir-cumstances, external support would be more appropriately used to developcore PFM skills – accounting, auditing, economic analysis – and PFMreform management skills, and to undertake diagnostic work, which mightraise awareness at the political level of the need for reform. lign support as closely as possible to the Government pro- Agramme and avoid pursuing independent technical assistanceinitiatives.In the country cases, externally financed support to PFMreform was most efficient and effective, when it directly financed, or sup-ported through technical assistance, actions and interventions identifiedwithin the Government PFM reform programme. The least efficient inter-ventions were those, which supported actions outside of the programme oronly tangentially related to it, and those interventions (such as institutionalsupport programmes), without explicitly defined outputs and outcomes.Thus, technical assistance and institutional support should focus on specif-ic outputs to which there is a shared commitment, and should be combinedwith Budget Support, where appropriate. nsure that aid policy and practise works in favour of the PFM Esystem and not against it.Aid dependent countries face the perpetualproblem of having to adapt their domestic PFM systems to the require-ments of their external aid partners. In the study countries – and else-where – significant problems have been created by aid mechanisms makingpartial use of government systems, or adopting special disbursement crite-ria for the use of the Government budget. Three particular problems arose,which undermined the good management of public finances in the studycountries: late disbursement of budget support tranches scheduled in theThetreasury/ cash flow plan for the 1stor 2ndquarter. imposition of special disbursement conditions or special report-Theing requirements for “basket funds” or “trust funds” managedthrough the national budget process. opening of special project accounts outside of the Single Treas-Theury Account. nsure that advice is up to date and informed by the experi-E ence within country, within the region and by wider interna-tional experience.External support can play a useful role in bringing tobear new and more widely informed perspectives on PFM problems, withwhich the Government is struggling. By opening “policy space” in this way,it can help to resolve problems but, when external advice is not wellinformed, it serves to close policy space and to perpetuate existing prob-lems. External agencies have a duty to ensure their advice is right, wherev-er possible, and, where this is not immediately possible, to ensure that they89

work jointly with Government to learn from initial mistakes until an ade-quate solution is found. nsure that internal procedures for the supervision and peerE review of initiatives to support PFM reform are effective inproviding acontinuous check on progress.Each of the case studycountries suffered from the continued implementation over several years ofinappropriate reform models and approaches. Policy advice will not alwaysbe right from the outset, in particular when working on PFM reform issueswhere a degree of experimentation is often necessary, but it is important toensure there are mechanisms in place to ensure mistakes do not go uncor-rected for too long. This requires the creation of a learning and adaptationculture, supported by a process of continuous evaluation. rovide support, where necessary, to regional institutions and Pprofessional associations working on PFM reform issues.In thecase study countries, both regional governmental institutions – such asWAEMU – and regional professional associations – such as CABRI andAfrosai – were found to be relatively influential in generating improvedpractises on public finance management. In so far as the scope of influenceof such bodies could be expanded by more substantial external support,then clearly such investments would be of benefit. However, it should berecalled that much of the value of these bodies derives from their ability topromote peer-to-peer learning: an excessive amount of external funding byDPs might undermine the effectiveness of this role. ontinue to provide support to CSOs and Legislative bodies onC PFM reform issues but accept that their influence may only beeffective in the longer term.The experience of the case study coun-tries suggests that advocacy work by CSOs and closer scrutiny by the Leg-islature are unlikely to have significant effects on the pace and content ofPFM reforms in the short to medium term. However, broader internationalexperience – including in the OECD countries – suggests that their influ-ence over the longer term may be important. Hence, support to such activi-ties should be continued but not as a substitute to direct support to theExecutive in the implementation of PFM reforms. In addition, the short tomedium term effectiveness of CSOs and the Legislature seem likely to beimproved by concentration on a narrower set of objectives, such as theimprovement of public access to fiscal information.

reform is fundamentally political – that is, to increase the efficiency andeffectiveness of public spending and, by implication, to pursue more effec-tively the development goals established for the national budget. It isa common mistake to perceive PFM reforms as purely “technical” meas-ures and this perception needs to be corrected so that there is wide politicalsupport for reform. In the first instance, this must start within the Execu-tive, with the Minister of Finance and his/ her team working closely withthe President and/ or Prime Minister to promote reforms and then widen-ing the scope of consultations to include the Cabinet and other members ofthe ruling party. In time, it should also be an objective to sensitise opposi-tion members to the need for PFM reforms, so as to ensure continuity inthe event of a change of Government. econdly, serious attention needs to be given to the design andS staffing of the structures established to coordinate and man-age PFM reforms.It is important that those responsible for coordinat-ing reforms should have both technical competence and authority. Themodel of a technical secretariat reporting directly to the Minister ofFinance is a good one. Such a model would normally work better thana secretariat attached to the Presidency or to the Prime Minister’s Office,whose legitimacy would commonly be brought into question by the seniorofficials of the Ministry of Finance. Another key feature of an effectivemodel is that authority for implementation should be retained at the levelof the relevant competent authority (the President of the Court of Audit,the Directors General of Treasury, the Budget, etc.) This will help to avoidany doubts over the responsibility for implementation and will ensure thatthe coordinating body is not over-burdened with both implementation andcoordinating/ monitoring responsibilities. hirdly, those responsible for coordinating PFM reformsT should exert firm control over external support to PFM andover dialogue and negotiations with Budget Support donors,related to PFM reform.A useful way to promote this is through theunification of responsibilities for attracting and managing external supportto PFM reform with those for coordinating implementation by the depart-ments and institutions of Government. This will help to ensure a strongalignment of external support to domestic priorities. In managing sucha model, Government staff needs to be prepared to be assertive in impos-ing Government priorities and ensuring that they are respected. ourthly, the structures established for monitoring PFMF reform should also promote learning from experience and thecorresponding adaption of implementation plans.PFM reform isinevitably complex and initial plans are likely to need adaptation andadjustment. If implementation of reform is to be efficient, the monitoringprocess must identify reform bottlenecks quickly and take speedy correctivemeasures. In order to ensure this happens effectively, management struc-tures must embody not only monitoring of progress but also periodic evalu-ation of performance. Evaluation work is best conducted by independentstakeholders; thus, arrangements for independent evaluation should be

built in from the outset. There are three models which have been common-ly adopted by successful PFM reformers: engagement of outside experts or groups of experts, who are con-thetracted to assess progress on an annual or two-yearly basis; creation of an internal team of “wise persons” or “eminent per-thesons”, of senior, generally retired civil servants and academics, whomight provide a periodic independent evaluation of performance;and creation and pre-funding of evaluation funds, so that morethefocused evaluation exercises can be quickly launched, based on com-petitive tenders. inally, the regular training of PFM staff needs to be aconsist-F ent priority.The most important aspect of this is to ensure a consistentoutput of people with core skills in auditing, accounting, economics, pro-curement and financial management. In many developing countries, therelated training bodies have deteriorated over time and commonly fail toproduce graduates of a sufficient number and quality. In such cases, invest-ment needs to be made to re-establish PFM training institutions of ade-quate quality, and to ensure their recurrent funding over time.

BibliographyANDREWS, M., (2009)Isomorphism and the Limits to African Public FinancialManagement Reform,Faculty Research Working Paper 09–012, Cambridge,MA: Harvard Kennedy SchoolANDREWS, M., (2010),How far have PFM reforms come in Africa?FacultyResearch Working Paper 10 –018, Cambridge, MA: Harvard KennedySchoolALLEN, R. (2009),The Challenge of Reforming Budgetary Institutions in DevelopingCountries,Washington: IMF.APRM (2004). Malawi and the African Peer Review Mechanism: A review ofnational readiness and recommendations for participation. Johannesburg,APRM.BARNETT, C., M. CHISVO, et al. (2006). Evaluation of DFID CountryProgramme: Malawi 2000 to 2005. London: DFID.BETLEY, M. BIRD, A. & GHARTEY, A. (2011)Ghana Country Case StudyReport.Oxford: Fiscus Limited and Mokoro Limited.BOESEN N. & D.DIETVORST (2007), SWAPS in Motion – Sector WideApproaches: from an Aid delivery to a Sector development perspective,Copenhagen & Brussels: Train4Dev.net.BOOTH, D. CAMMACK, D. ET AL (2006), “Drivers of Change andDevelopment in Malawi,”ODI Working Paper 261,Overseas DevelopmentInstitute, London.BOOTH, D.CROOK, R., GYIMAH-BOADI, E., KILLICK, T. & R.LUCKHAM (2004),Drivers of Change in Ghana: Overview report,ODI, Lon-don & CDD, and Accra.CAIDEN, N. & WILDAVSKY, A. (1980)Planning and budgeting in poor countries,Piscataway, NJ.CHICHE, M. (2011)Evaluation of PFM Reforms in Burkina Faso: Desk Report,Oxford: Fiscus Limited and Mokoro Limited.COONEY, G., A. WENDEROTH, et al. (2010). CIDA Malawi Country Pro-gramme Evaluation, 2000 -- 2008. Gatineau, Quebec, CIDA.DE RENZIO, P. ( July 2009)Approach & Methodolog y for the Evaluation of DonorSupport to PFM Reform in Developing Countries:Part A, Danida, Copenhagen.

DE RENZIO, P. (2011), “Buying better governance: the political economy ofbudget reform in aid-dependent countries,”Global Economic GovernanceWorking Paper,Oxford University.DE RENZIO, P. & DOROTINSKY, W. (2007)Tracking Progress in the Quality ofPFM Systems in HIPCs: An update on past assessments using PEFA data.Wash-ington, DC, PEFA Secretariat.DE RENZIO, P., M. ANDREWS and Z. MILLS (November 2010),Evalua-tion of Donor Support to PFM Reforms in Developing Countries: Analytical Study ofquantitative cross-country evidence,Overseas Development Institute, London.DUREVALL, D. and M. ERLANDSSON (2005). Public Finance Manage-ment Reform in Malawi. Sida Country Economic Report 2005:1. Stock-holm: Sida.FARAGHER, R. (2011),Evaluation of PFM Reforms in Malawi:Desk Report,Oxford: Fiscus Limited and Mokoro Limited.FOLSCHER, A., MKANDAWIRE, A. and FARAGHER, R. (2011)MalawiCountry Case Study Report,Oxford, Fiscus Limited and Mokoro Limited.FYSON, S. (2009),Sending in the Consultants: Development Agencies, the private sectorand the reform of public finances in low-income countries,International Journal ofPublic Policy Vol. 4, pp314–343.GEORGE, A.L and A. BENNETT (2005), Case studies and theory develop-ment in the social sciences. Cambridge, MA: MIT Press.GORDON, A and M.BETLEY (2011)Evaluation of PFM Reforms in Ghana: DeskReport,Oxford: Fiscus Limited and Mokoro Limited.GRINDLE, M. “Good Enough Governance Revisited”,Development PolicyReview(2007), Overseas Development Institute, London.IEG (2008)Public Sector Reform: What Works and Why? An IEG Evaluation of WorldBank Support. Washington,DC, World Bank, Independent EvaluationGroup.JENKINS, K. (2006), Politicians & Public Services: Implementing Change ina Clash of Cultures, London: Edward Elgar Publishing.KEEFER, P. (2007), “Political Economy” in WORLD BANK,Ghana: Meetingthe Challenges of Accelerated & Shared Growth – Country Economic Memorandum,Volume III,Washington DC: World Bank.LAWSON, A. (2011).Joint Evaluation of PFM Reforms in Burkina Faso, Ghana andMalawi: Inception Report.Oxford, Fiscus Limited & Mokoro Limited.LAWSON, A., CHICHE, M & OUEDRAOGO, I. (2011).Burkina Faso Coun-try Case Study Report.Oxford, Fiscus Limited & Mokoro Limited.

LAWSON, A., DE RENZIO, and P. (September 2009),Approach & Methodol-og y for the Evaluation of Donor Support to PFM Reform in Developing Countries:Part B,Danida, Copenhagen.LINDESMITH, A. (1968)Addicts and Opiates.Chicago: AldineOECD-DAC (2002, reprinted 2010),Glossary of Key Terms in Evaluation andResults-Based Management,OECD-DAC, ParisPAWSON, R. (2000)Middle-Range Realism,Archive Européennes des Sociolo-gie XLIPAWSON, R & N. TILLEY (1997)Realistic Evaluation.London: SagePEFA (2005)Public Financial Management Performance Measurement Framework.Washington, DC, Public Expenditure and Financial Accountability(PEFA) Secretariat.POPPER, K. (1959)The Logic of Scientific Discovery,London: Hutchinson.PRETORIUS, C. & PRETORIUS, N. (2008)A Review of PFM Reform Litera-ture.London, DFID.PRITCHETT, L., WOOLCOCK, A & ANDREWS, M., (2010)CapabilityTraps? The Mechanisms of Persistent Implementation Failure,Washington DC:Centre for Global Development Working Paper 234.RAKNER, L., L. MUKUBVU, et al. (2004).The budget as theatre – the formaland informal institutional makings of the budget process in Malawi.Bergen: CMI.ROBINSON, M. (2007) The Politics of Successful Governance Reforms: Les-sons of Design and Implementation.Commonwealth & Comparative Politics,45, 521–548.SCHICK, A. (1998)A Contemporary Approach to Public Expenditure Management,Washington, World Bank.SCOTT G. (2001),Public Sector Management in New Zealand: Lessons & Challenges,Wellington.WESCOTT, C. G. (2008)World Bank Support for Public Financial Management:Conceptual Roots & Evidence of Impact, IEG Background Paper,And WashingtonDC: World Bank.WILDAVSKY, A. B. (1975)Budgeting: a comparative theory of budgetary processes,Piscataway, NJ, Transaction Publishers.WORLD BANK (2008).Public Sector Reform: What Works and Why? An IEGEvaluation of World Bank Support,Washington, DC: World Bank, Independ-ent Evaluation Group.95

Annex One: Biographical detailsof Finance MinistersTable 9. Burkina Faso: Ministers of Finance 1996–2011501996–1997m. Kadré ouédraogo:formerly executive secretary of ECOWAS (1985–1993);Governor at BCEAO (1993–1996)Became Prime Minister (1997–2000); now Ambassador in Brussels1997–2001m. tertius Zongo:formerly Deputy Minister in Charge of Budget & Planning(1995–1997); Director General of Cooperation (1992–1995); Chief of Depart-ment of Multilateral Cooperation (1988–1992).later became Ambassador to the uS, and Prime Minister (2007–April 2011)m. Jean-Baptiste Compaoré:formerly Deputy Minister for Finance & Budget(2000–2002); General Secretary at the Office of the Prime Minister (1996–2000); Bank inspector at the WAEMU (1990–1995); Advisor at the cabinet ofthe Presidency (1995–1996); Executive at the BCEAO (1981–1990)later became First deputy Governor and currently interim Governor attheBCEAO2008-to datem. Lucien Bembamba:formerly Deputy Minister in charge of the Budget(2007–2008); President of the National Public Debt Committee (2006); formerDirector General of Treasury and Public Accounting (1993–2007); Executive ofthe BCEAO (1982–1993)

Dr. Cassim Chilumpha:former Lecturer at the Polytechnic (University of Ma-lawi). Served as Minister under President Bakili Muluzi from March 1994 to2000, then from 2003 to 2004, and again from 2004 to 2006 under PresidentMutharika, serving as Minister for: Defence, Education, Justice, Finance, Ed-ucation, Statutory Corporations, and Water. Minister of Finance from 1998 to2000.1998–2000later became vice President (2004–2009); independent Member of Parlia-ment since September 2009Friday Jumbe:former General Manager of Agricultural Development & Mar-keting Corporation (ADMARC). Later Member of Parliament June 2004 toMay 2009.now interim leader of the united democratic FrontGoodwill Gondwe:former General Manager of the Reserve Bank of Malawi;senior Vice President and Acting President of the African Development Bank;Senior Advisor, Director for Africa and Special Advisor to the Managing Direc-tor of the International Monetary Fund; and Chief Economic Advisor to theformer Head of State.2004–2009later appointed as Minister of local Government and dropped from Cabinet injune 2009. Currently appointed as Minister of Energy from August 2011Ken Kandodo:former auditor with KPMG. Elected as Member of Parliamentfor Kasungu Central Constituency in May 2009. Appointed as Minister of Fi-nance on June 15, 2009.now just aMember of Parliament for kasungu Central ConstituencyDr Ken Lipenga:former Lecturer at the University of Malawi and former Min-ister of Labour and Vocational Training (2005–2006); Deputy Minister of Fi-nance (2006–2008); Minister of Economic Planning & Development (2008). Hewas then dropped from Cabinet, before being appointed Minister of Financein August 2011.he is Member of Parliament for Phalombe East since May 1997

hon Kwadwo Baah Wiredu:Chartered Accountant by profession and formerMember of Parliament (1997–2008); worked in various positions with publicinstitutions; senior consultant in private firms; Minister of Local Governmentand Rural Development (2001–2003); Minister of Youth and Sports (2003–2005).died on 24 September 2008 from ill health.Dr. Anthony Akoto osei:Economist by profession (monetary and applied eco-nomics); consultant at the Institute of Urban Affairs, USA; lecturer (1985–1987); Deputy Minister of Finance and Economic Planning (2003–2008); ap-pointed Minister of Finance (Oct– Dec 2008) on the death of Hon KwadwoBaah WireduBecame Member of Parliament in 2005 to dateKwabena Duffour:Economist, Finance, Banking and International Financeexpert. Worked as afull time banker; Deputy Governor of Bank of Ghana(1995–1997); Governor of Bank of Ghana (1997–2001).

Annex Two: Quantitative Studysample of 100 CountriesDate and status of PEFA reports. ’Final-P’ denotes reports thathave been made public on the PEFA website.

Country

123456789101112131415161718192021222324252627282930313233343536AfghanistanAlbaniaArmeniaAzerbaijanBarbadosBelarusBelizeBeninBoliviaBotswanaBrazilBurkina FasoBurundiCambodiaCameroonCape VerdeCentral African RepublicChadColombiaComorosCongo, Dem. Republic ofCongo, Republic ofCote d’IvoireDominicaDominican RepublicEgyptEl SalvadorEthiopiaFiji IslandsFYR MacedoniaGambiaGeorgiaGhanaGrenadaGuatemalaGuinea

Date

Jun. 08Jul. 06Oct. 08Jan. 08Oct. 06Apr. 09Oct. 08Sep. 07Aug. 09Feb. 09Oct. 09Apr. 07Feb. 09Mar. 10Jan. 08Dec. 08Jun. 08Jul. 09Jun. 09Jan. 08Mar. 08Mar. 06Nov. 08Apr 07Nov. 09May, 09May 09Oct. 07Jun. 05Aug. 07Mar. 09Nov. 08Jan. 10Mar. 10Mar. 10Jul. 07

Status

Final-PFinal-PFinal-PFinalDraftFinalDraftFinal-PDraftFinal-PDraftFinal-PFinalDraftFinalFinal-PFinalFinalDraftFinalFinalFinal-PFinal-PDraftFinalDraftFinal-PFinal-PFinalFinal-PDraftFinal-PFinalFinal-PDraftFinal

Annex Four:Consultant Terms of Reference1. IntroductionThese Terms of Reference are for case studies to be carried out in BurkinaFaso, Ghana and Malawi on the reform of public financial management(PFM) systems, the results achieved, the role played by donors and otherinstitutional and contextual factors that may contribute to or hinder PFMreform outcomes. The case studies will build on empirical analysis thatinvestigates where and why PFM reform has delivered results and, con-versely, where and why it has not. The main focus of the case studies willbe to investigate whether and how donor behaviour and the design andimplementation of PFM reform makes adifference to the achievement ofresults, or whether other domestic contextual factors carry more weight.

2. BackgroundThe Paris Declaration on Aid Effectiveness and the associated emphasison the use of country systems, budget support, and governance and anti-corruption have triggered increased attention on the reform of PFM.Strong PFM systems are akey element of the institutional and governanceframework needed for building peaceful and stable societies and successfuleconomic and social development, essential to improved service deliveryand to the achievement of the Millennium Development Goals.Nevertheless, PFM systems in many developing countries remain weak andthere is lack of certainty or consensus on the role of donors and the contextunder which external support can best assist the process of PFM reform.To address this, the evaluation departments of DANIDA (Denmark), Sida(Sweden) and the AfDB (African Development Bank) have agreed, in con-sultation with the OECD-DAC Evaluation Network, to manage ajointevaluation of PFM reforms in developing countries. This and otherplanned joint evaluations, including the joint evaluation of the impact ofbudget support, public sector governance reform, support to anti-corrup-tion programmes, and the implementation of the Paris Declaration,willfeed into discussions prior to the next High Level Forum on Aid Effective-ness (HLF-4) in Busan (29 November– 1December 2011).The PFM evaluation is interested in finding answers to two related ques-tions:(a) Where and why do PFM reforms deliver results (i.e. improvement inthe quality of budget systems); and(b) Where and how does donor support to PFM reform efforts contributemost effectively to results?

To answer these questions the evaluation design is made up of severalcomponents outlined in summary below.First,analytical background workhas been undertaken both to: efine what is meant by PFM reform results; andd  consider how results can be measured across countries and overtime to assess the degree to which change in the quality of PFM sys-tems has occurred (see Lawson/De Renzio Approach and Method-ology for the Evaluation of Donor Support to PFM in DevelopingCountries Part AJuly 2009 and Part BSeptember 2009).Second aliteraturereviewhas been completed looking at therange of approaches to PFM reform, donor support and existing evi-dence on success/failure of PFM reform approaches1.Third, aquantitativeanalysishas been undertaken to identifycountries where PFM reform has delivered results in the quality ofPFM systems; where it hasn’t, and the contextual factors that mightexplain these differences as well as the correlation with donor support2.Fourth,country case studieswill follow up the findings from thequantitative analysis and explore why, in some cases, donor supportappears correlated with results, and why in others it does not. Thecase studies will explore whether and how donor behaviour and theapproach to PFM reform design and implementation makes adiffer-ence to results. Five case studies are planned in Sub-Saharan Africastarting with Burkina Faso, Ghana and Malawi.Finally, aregional Africa synthesis reportwill be compiledthat will bring together findings from each of the evaluation productsoutlined above.Country case studies have been selected on the basis of data availability(see below) and because they provide examples where budget institutionsimproved with: (i) high donor effort; (ii) low donor effort; and whereregardless of donor effort, budget institutions did not improve.

4. Scope and LimitationsCountry case studies cover the period from 2001–2009.3 The period rep-resents atime in which donors became increasingly interested in PFMand agreed to increase the effectiveness of aid expenditure, including byusing country systems to channel and deliver aid finance.As PFM performance information is only widely available for centralgovernment, the scope of the case studies is restricted to central govern-ment organisations. Nevertheless, country visits may provide an opportu-nity to gather information (in addition to that contained in PEFA) on theextent to which PFM reforms are beginning to extend beyond centralgovernment institutions to local government and to service providers.The extent to which the government is taking alead in this may indicateownership and reform sustainability.The quantitative analysis found apositive and significant, albeit weak,correlation between donor support to PFM reforms and improvements inPFM systems. It also found some positive correlations between the wayaid is provided and the strength of PFM systems. However, these averageeffects cannot be taken as causal and universal, and need to be furtherinvestigated. Therefore the main purpose of the country case studies is tounpack the nature of PFM reform in different cases where there is foundto be: (i) apositive correlation with donor support; (ii) anegative correla-tion with donor support; and (iii) no correlation between PFM results anddonor support. As such, case studies will focus on the history of PFMreform inputs; what has been provided, for what purpose, in whatsequence, for how long and at what cost that might help to explain thecorrelation (positive or negative) with PFM results or lack of them. Thecase studies will therefore not investigate the impact of PFM reform (par-ticularly on service delivery) but will instead focus on inputs in the evalu-ation framework; how they have been identified, designed and deliveredand the significance of this for the delivery of intermediate outcomes(explained in detail below). In countries where there has been high donorsupport to PFM, akey line of enquiry is the extent to which the applica-tion of aid effectiveness principles is found to make adifference to results.However, given the range of factors that contribute to PFM results, itmay be difficult to directly attribute results to donor support.4PFMreform interventions are treated as inputs in the evaluation frameworkand the case study methodology is centred on the ability to assess theinstitutional and contextual factors that helped to support success and/orfailure of these inputs at each stage of the evaluation framework. It mayalso be possible to link intermediate outcomes to outputs and donorinputs. For example, one dimension of PFM reform– linking policy toplanning and budget (an intermediate outcome in the evaluation frame-work)– may receive substantial donor support because while it is difficult34Or the date of the second PEFA report, which might be earlier than 2009.In fact, the quantitative analysis highlights how economic factors in particular explainalarge part of variation in the successful implementation of PFM reforms.

from atechnical perspective it may be relatively easier from apoliticalperspective. Donor support for the achievement of the other intermedi-ary outcomes– greater transparency and comprehensiveness and con-trol, oversight and accountability– may not be as strong because thepolitical costs of these reforms may be higher despite their relative techni-cal ease. It will therefore be important for the evaluation case studies toexplore the wider context of reform intervention and whether certainreforms are pursed because they are politically more palatable than oth-ers. In other words, the case studies would examine the extent to whichdonor support is concentrated at particular phases of the budget cycle.While the evaluation framework identifies final outcomes including theoperational efficiency of public spending, it may be too soon to draw con-clusions about the impact and sustainability of results. This is largely dueto the fact that the evaluation period is relatively short and while qualityPFM is necessary for the quantity and quality of service delivery, it is notsufficient. Nevertheless, it may be useful to consider what factors help tosupport on going reform and what factors risk sustainability e.g. is PFMreform supported by civil society, the Parliament and the business com-munity; implemented in astable and growing economy; building anddeveloping the capacity of the Ministry of Finance as akey central gov-ernment body; and spreading further than the centre to include sectorMinistries, local government and service delivery units; and supportedby on going donor support and technical assistance?

5. Methodology for the Selection of Case StudiesCase study countries have been selected on the basis of data availability.All case studies have at least two Public Expenditure and FinancialAccountability (PEFA) assessments available (which covers aperiod of atleast three years) plus World Bank and IMF HIPC assessments whichextends the evaluation period by at least another four years (or moredepending on the date of the HIPC assessments) which may be sufficientto observe changes in the quality of PFM systems.The table below separates the 14 countries in Sub-Saharan Africa that meetthe data requirements, into countries where reforms have delivered results(i.e. an increase in HIPC/PEFA scores between 2001 and 2007– or the dateof the second PEFA assessment– and countries where reforms did not bringabout any improvement or where the quality of PFM systems deteriorated.PEFA Performance in 14 African Countries 2001–2007

Countries where budget institutions

improved

Burkina Faso, Ethiopia, Ghana, Mali,Tanzania, Zambia

Countries where budget institutions

did not improve or deteriorated

Benin, Guinea, Madagascar, Malawi,Mozambique, Rwanda, Sao Tome andPrincipe, Uganda

Of these countries, case studies will be selected so that three types of situ-ation can be examined:One in which donor support appears to be positively correlated withPFM improvement;One in which donor support appears to be negatively correlated withPFM improvement; andOne in which significant PFM improvements appear to haveoccurred despite relatively low levels of donor support.This suggests that countries are selected from across the following table.Relative Impact of Donor Support to PFM reforms in SSA (1998–2007)

PFm reform

Countries where

Budget Institutions

Improved

Burkina Faso, Tan-zania, ZambiaEthiopia, Ghana,Mali

Budget Institutions did not im-

prove

Benin, Malawi, Mozambique,Rwanda, Sao Tome and Princi-pe, Uganda

6. Analytical ApproachThe first step in answering the related questions of where and why doPFM reforms deliver results and where and how does donor support con-tribute most effectively to results, is to build acommon definition of whatis meant by results. The second step is to identify empirical informationthat might help to measure results and compare them across countriesand over time (see Assessing Budget Institutions and Budget Reforms inDeveloping Countries: Overview of theoretical approaches and empiri-cal evidence. Paolo de Renzio July 2009).For the purpose of this evaluation, the following three dimensions ofbudget institutions provide abasis for assessing their overall quality:Transparency and comprehensiveness:looks at issues related tothe quality of budget information, from the classification system to thecoverage and clarity of budget documents; accessibility to budget infor-mation by the Legislature, the general public, media and civil society

Linking budgeting, planning and policy:assesses the extent towhich the budget is effective in converting policy objectives into rel-evant taxation and spending actions; budgets are derived from accu-rate medium term forecasts and contain apolicy perspectiveControl, oversight and accountability:considers whether ade-quate mechanisms are in place to promote overall accountability forthe use of public resourcesQuality budget institutions are defined as those that exhibit higherdegrees of transparency; policy orientation and control/accountability(see Table 1 below). At the opposite end, weak budget institutions areidentified by their opacity, their lack of linkages with planning and poli-cy, and the absence or weakness of mechanisms for monitoring andaccounting for the use of public funds.These dimensions are consistent with parts of the PEFA assessment aswell as with indicators developed for HIPC assessments (Table 1).

Unsurprisingly, there are limited sources of information and cross-countrydata which can be relied on to assess and compare the quality of budgetinstitutions. The most comprehensive attempt at constructing aframeworkto assess the quality of budget institutions is the PEFA PFM PerformanceMeasurement Framework (PEFA 2005) based on 31 indicators, which cov-er institutional arrangements at all phases of the budget cycle. Moreover,the framework contains all the information needed to measure the qualityof budget institutions long the three dimensions identified above.The country case studies will explore the extent to which PFM reform ismore likely to produce results when there is an enabling environment for

reform, when donor behaviour follows the principles of aid effectiveness, andwhen PFM reform interventions follow certain principles.5This suggests thatPFM reform is more likely to produce results in the following circumstances:Economic Growth and Political Stability:PFM reforms takeplace in astable environment that allows for the time, policy spaceand flexibility needed to implement complex governance reforms,and the additional public funds generated by growth.Reform Planning and Design:PFM reform inputs consider thelocal context taking into account the strength of existing institutions.Reform plans have been prioritized and sequenced to implementbasics first and do not overwhelm existing administrative capacity.Strengthened Approach:reforms are country owned and man-aged through existing processes; with donor support harmonized andaligned behind country led reform programmes and aid is channeledthrough country PFM systems6.Political Economy:PFM reforms have sustained high level politi-cal support for governance reforms in general (including civil servicereform) and reflect political priorities and feasibility; political econo-my factors (such as patronage networks) are less powerful.Demand side governance:PFM reforms build on existing publicdemand for improved PFM through strengthening transparency ofdecision making and financial information, and there is greateraccountability to the public and users of public services. Countrieswhere these processes exist are more likely to deliver results in PFMreform, but this could also be anecessary pre-condition to more dif-ficult or politically sensitive PFM reforms.The relevance of these factors to the PFM change process has beenincorporated into an evaluation framework. The purpose of the pilotcountry case studies is to test the evaluation framework and to elaboratefurther on where and why PFM reforms deliver results and how donorsupport can more effectively support the PFM change processes.

7. Evaluation QuestionsDetailed questions for each country case study regarding PFM reformsare listed below against the OECD DAC evaluation criteria of relevance,efficiency and effectiveness of PFM reform. The questions have beenstructured to reflect the theory of change set out in the evaluation frame-work in Annex 1. At each level of the evaluation framework (inputs, out-5See Paolo de Renzio ( July 2009): PFM Evaluation Approach Paper, Part Achapter 6:“Explaining success in budget reforms: lessons from the political economy of govern-ment reforms.” The theory of change largely draws on the lessons learnt from firstgeneration structural adjustment reforms, rather than second generation governancereforms. However, the theory closely suggests that the principles of aid effectivenessalso apply to PFM reform.While this may appear tautological, there can often be several PFM reform projectinterventions and TA initiatives (World Bank, IMF, ADB and so on) operating within oneinstitution, usually the Ministry of Finance with no single agreed strategy for PFM reform,

puts, intermediate and final outcomes) the evaluation will consider theinstitutional and contextual factors that influenced the design and imple-mentation of PFM reform and their significance in delivering reform suc-cess.7It is important to note that these questions refer to all PFM reforminputs whether or not they are financed by external donors.How relevant is PFM reform to local context and existingsystems? s there a government led PFM reform programme that has highI level political support? oes PFM reform respond to domestic priorities, e.g. politicallyD driven public sector reform agendas, macroeconomic and fiscalneeds, political priorioties for improved service delivery? s donor support designed and structured to support governmentI led and government managed initiatives? o PFM reform programmes include a component aimed atD strengthening budget reporting e.g. to the public. Or do PFMreform programmes include components to include the public inresource allocation decisions? s external support to PFM reform designed to fit with the natureI of political support for reform, to the institutional strengths andweaknesses of the existing PFM system, and to the organisationalcapability of the lead agencies (e.g. finance ministry) in PFMreform? Are international models of PFM reform transplanted ona “one-size fits all” basis or is PFM reform developed incremental-ly to fit with existing administrative capacity?8 re PFM reforms consistent with on-going public administrationA reforms? s donor support based on building existing PFM systems ratherI than creating new ones? s there evidence that donor supported reforms have overwhelmedI existing institutional capacity? hat is the role of technical assistance in PFM reform design?W  re PFM reform and management processes supported by orA include active consultation and communication with a wide rangeof stakeholders involved in the reforms, as well as active measuresto broaden support for reform?How efficient and cost effective is PFM reform? fficiency should look at the ratio between costs and output orE outcomes. It will be important to estimate what PFM reformcosts? How much has been spent by Government initiatives and

by donors e.g. on personnel and equipment, to achieve particularPFM objectives? s donor support for PFM reform coordinated around a singleI PFM reform plan or strategy or is support fragmented across sev-eral initiatives? ave donor efforts been slow at getting started or taken longerH than expected, requiring on-going TA support? s donor support reliant on specifically designed PFM reformI management units (project implementation units)?How effective is PFM reform? s there any additional evidence of PFM reform that is not cap-I tured in the PEFA framework e.g. reforms extending beyond cen-tral government institutions? ave PFM reforms (including donor support to reforms) movedH beyond de jure reform aspects, such as approving laws and regula-tions, to de facto aspects, such as changes in actual budget prac-tices, and have these elements of PFM systems improved? ave PFM reforms extended beyond the centre (e.g. CentralH Finance Agencies) to include, for example, sector Ministries, localgovernment and service delivery units and what explains thisspread? Have reforms been effective in improving PFM perfor-mance beyond central finance agencies? n aid dependent countries, to what extent has the use of generalI budget support, PFM-related conditionality, and efforts to reduceaid fragmentation contributed to strengthening PFM perfor-mance? Have these efforts impacted across all aspects of PFM, oron specific areas such as de jure and concentrated PFM processes? re country systems for financial reporting and accountabilityA utilised by donors? o what extent is aid expenditure included in different stages ofT the budget process9?

Task 2: Desk ReviewReview existing PEFA/HIPC assessments and performance data (for thethree case study countries) – what does the evidence tell us about PFMreform progress over the evaluation period? What specific reforms havebeen followed and is it possible to track them over time?Gather information on donor support to PFM reform over the evaluationperiod (including project documents where these are available). While 2001is the date of the first HIPC assessment, it might also make sense to extendthat period backwards to capture earlier donor PFM support, and initialreforms that took place in the mid to late 1990s.Identify the level and character of government initiatives, and (changes in)the level of political will to undertake PFM reforms during the evaluationperiod, classifying it according to the following categories: tage of the budget cycle (e.g. preparation, approval, execution, audit)S  udgeting time horizon (e.g. annual budget, MTEF)B  nvolved stakeholders (e.g. Parliament, CSOs, DPs)I  ype of input (e.g. legislative, human capital, infrastructure)T  ost (direct and indirect) and timeC Identify donor support (inputs in the evaluation framework) to PFM reformduring the evaluation period in each country case study and classify itaccording to the three dimensions of quality PFM (described above) andthe following: ts phase in the budget cycle e.g. preparation, approval, execution, auditi  ype e.g. technical assistance, training, capacity building, softwaret and computer installation, budget support, dialogue on PFM reform; rocess of delivery e.g. project management unit, or through Govern-p ment systems; onor providing the support e.g. World Bank, IMF, AFDB, and bilat-d erals and whether it is joined up or implemented through separateproject agreements; ost and time;c Using project documents, identify the outputs that support interventions areintended to deliver e.g. people, skills and organizational capacity; changes inlaws, rules and procedures; improved information systems and business pro-cesses; and changes in incentives and controls (see evaluation framework);Compile a timeline of support showing the sequencing of donor supportedreform activities (plus investments). Can results in PEFA assessments belinked to specific reforms that have been supported by donors?10 Lawson/De Renzio Approach and Methodology for the Evaluation of Donor Supportto Public Financial Management (PFM) in Development Countries Part A July 2009and Part B September 2009

Task 3: Undertake country visitsThe purpose of the country visits is to assess the relevance, efficiency, effec-tiveness and sustainability aspects of the theory of change framework and howdonor support has been designed and implemented.The evaluation team would be expected to carry out structured and semistructured interviews with Ministry of Finance officials, other officials in gov-ernment, sector ministries, local government, politicians, civil society anda range of donors both those involved in PFM reform and those that are not.The consultants could consider sending a limited number of questions inadvance of the country visit to both donors and government officials. If a jointdonor/government budget support or PFM group exists detailed discussionshould be held with these groups to review experience over time.Task 4: Report draftingThe consultants will be expected to produce stand alone reports for each coun-try case study. Reports should be no longer than 30 pages with additional infor-mation included as annexes as necessary. Reports should be succinct as it isimportant to produce written information which is accessible to a wide audienceand to readers whose first language may not be English (or conversely French).As far as possible, the initial findings of the evaluation should be discussed withthe participating government, with donor partners and other stakeholders incountry for comment and feedback before the evaluation team departs. Draftreports would be presented to the Management Group and Evaluation Refer-ence group for comments and feedback before the final report is produced.

11. Consultant qualifications and skills10.1 The work will require a small team of consultants who have experiencein PFM reform, with part of that experience being in the evaluation ofdevelopment policy, programs or project operations. Support may berequired to gather information on donor support in both case studiesincluding project documents, PEFA reports and HIPC data and to pro-vide support with the desk study. Two consultants will be required foreach case study and it will be important that the team leader is fluent inboth English and French (in reading and writing) and involved in allthree case studies. Consultants would be expected to be familiar withreform approaches, partner government initiatives and interventions ofdonor agencies in African countries particularly in the area of PFM.Familiarity with PEFA assessment systems will be important.Compulsory requirements for personnel are specified in section 4.2.1 for teamleader and 4.2.2 for other personnel. Evaluation criteria for qualification andcompetence are specified in 7.2.1 for the team leader and 7.2.2 for other teammembers.Final reports should be submitted in both English and French. All reportssubmitted should be professionally edited.

joint Evaluations1996:1The international response to conflict andgenocide: lessons from the Rwanda experience:Synthesis ReportJohn Eriksson, Howard Adelman, John Borton, Krishna Kumar,Hanne Christensen, Astri Suhrke, David Tardif-Douglin, SteinVillumstad, Lennart WohlgemuthSteering Committee of the Joint Evaluation ofEmergency Assistance to Rwanda,1996.1997:1Searching for Impact and Methods: NGOEvaluation Synthesis Study.Stein-Erik Kruse, Timo Kyllönen, Satu Ojanperä, Roger C. Rid-dell, Jean-Louis VielajusMin of Foreign Affairs Finland, OECD-DAC, Sida, 1997.1997:2Measuring and Managing Results: Lessons forDevelopment Cooperation: Performance ManagementDerek PoateUNDP/OESP Sida, 1997.2003:1Local Solutions to Global Challenges: Towards Effec-tive Partnership in Basic Education. Final Report.Joint Evaluation of External Support to Basic Educa-tion in Developing Countries.Ted Freeman, Sheila Dohoo FaureNetherlands Ministry of Foreign Affairs, CIDA, DFID, Depart-ment for Foreign Affairs Ireland, EU, BMZ, JICA, Ministry ofBasic Education and Literacy Burkina Faso, Danida, Norad,Sida, UNESCO, UNICEF, World Bank. 2003.2003:2Toward Country-led Development : a Multi-PartnerEvaluation of the Comprehensive Development Frame-work : Synthesis reportCarol Lancaster, Alison Scott, Laura Kullenberg, Paul Collier,Charles Soludo, Mirafe Marcos, John Eriksson, Alison Scott;Ibrahim Elbadawi;John Randa,World Bank, OED, CIDA, Danida, Norad, ODI, JICA, Sida, 2003.115

Tsunami Evaluation Coalition (TEC) Actionaid, AusAID, BMZ,CIDA, Cordaid, Danida, Dara, Irish Aid, DFID, FAO, IFRD,Federal Min for Economic Cooperation and Development Germa-ny, JICA, Min des Affaires Étrangères France, Min des AffairesÉtrangères Luxembourg, Norad, NZAID, DEZA, Sida, UN,UNDP, UNFPA, Unicef, Usaid, WFP, WHO, World Vision, 2006.2006:3Coordination of International Humanitarian Assis-tance in Tsunami-affected countriesJon Bennett, William Bertrand, Clare Harkin, Stanley Samaras-inghe, Hemantha WickramatillakeTsunami Evaluation Coalition (TEC) Actionaid, AusAID, BMZ,CIDA, Cordaid, Danida, Dara, Irish Aid, DFID, FAO, IFRD,Federal Min for Economic Cooperation and Development Germa-ny, JICA, Min des Affaires Étrangères France, Min des AffairesÉtrangères Luxembourg, Norad, NZAID, DEZA, Sida, UN,UNDP, UNFPA, Unicef, Usaid, WFP, WHO, World Vision, 2006.2006:4Funding the Tsunami Response: A synthesis of findingsMichael Flint, Hugh GoyderTsunami Evaluation Coalition (TEC) Actionaid, AusAID, BMZmCIDA, Cordaid, Danida, Dara, Irish Aid, DFID, FAO, IFRD,Federal Min for Economic Cooperation and Development Germa-ny, JICA, Min des Affaires Étrangères France, Min des AffairesÉtrangères Luxembourg, Norad, NZAID, DEZA, Sida, UN,UNDP, UNFPA, Unicef, Usaid, WFP, WHO, World Vision, 2006.2006:5Links between relief, rehabilitation and developmentin the Tsunami response: A synthesis of initial findingsIan ChristoplosTsunami Evaluation Coalition (TEC) Actionaid, AusAID, BMZmCIDA, Cordaid, Danida, Dara, Irish Aid, DFID, FAO, IFRD,Federal Min for Economic Cooperation and Development Germa-ny, JICA, Min des Affaires Étrangères France, Min des AffairesÉtrangères Luxembourg, Norad, NZAID, DEZA, Sida, UN,UNDP, UNFPA, Unicef, Usaid, WFP, WHO, World Vision, 2006.2006:6The role of needs assessment in the Tsunami response –Executive summaryClaude de Ville de Goyet, Lezlie C MorinièreTsunami Evaluation Coalition (TEC) Actionaid, AusAID,BMZm CIDA, Cordaid, Danida, Dara, Irish Aid, DFID, FAO,IFRD, Federal Min for Economic Cooperation and Develop-ment Germany, JICA, Min des Affaires Étrangères France, Mindes Affaires Étrangères

Luxembourg, Norad, NZAID, DEZA, Sida, UN, UNDP, UNF-PA, Unicef, Usaid, WFP, WHO, World Vision, 2006.2006:7Evaluation of Coordination and Complementarity ofEuropean Assistance to Local Development: with Ref-erence to the 3C Principles of the Maastricht TreatyRobert N. LeBlanc and Paul BeaulieuSida, Ministry for Foreign Affairs, Austria, Ministry for ForeignAffairs, Department for International Development Cooperation.Belgium, Min. des Affairs étrangères/Direction General de laCooperation International, France, Department of Foreign AffairsDevelopment Co-operation Division, Ireland and Ministry of For-eign Affairs/Directorate-General for International Cooperation, theNetherlands, 2006.2007:1Evaluation of General Budget Support – Note onApproach and Methods. Joint Evaluation ofGeneral Budget Support 1994–2004AFD, DFID, MOFA, NZAID, USAID, AusAID, BMZ, JBIC,NORAD, Danida, SECO, CIDA, JICA, Min of Foreign AffairsSpain, Portuguese Development Cooperation, Sida, 2007.2007:2Evaluating Co-ordination, Complementarity andCoherence in EU development policy: a synthesisEvaluation Services of the European Union, Sida, Ministry forForeign Affairs, Austria,Ministry for Foreign Affairs, Department for InternationalDevelopment Cooperation. Belgium, Min. des Affairsétrangères/Direction General de la Cooperation International,France, Department of Foreign Affairs Development Co-opera-tion Division, Ireland and Ministry of Foreign Affairs/Directo-rate-General for International Cooperation, Netherlands, 2007.2007:3Evaluating Democracy Support: Methods and Experi-ences.Sida, Department for Evaluation and Internal Audit and Internation-al Institute for Democracy and Electoral Assistance (IDEA), 2007.2007:4Peer Review Evaluation Function at the World FoodProgramme (WFP). Peer Panel Members: Jock Baker, Stefan Dahl-gren, Susanne Frueh, Ted Kliest, Zenda Ofir.Advisors to the Pan-el: Ian Christoplos, Peta Sandison Sida, BMZ, UNEG, WFP, 2007.2008:1118

Anneke Slob, Alf Morten JerveSida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.2008:1:1Managing Aid Exit and Transformation:Summary of a Joint Donor EvaluationJesper HeldgaarSida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.2008:1:2Managing Aid Exit and Transformation: India Coun-try Case StudyAlbert de Groot, CK Ramachandran, Anneke Slob, Anja Wil-lemsen, Alf Morten JerveSida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.2008:1:3Managing Aid Exit and Transformation: South AfricaCountry Case StudyElling N Tjønneland, Pundy Pillay, Anneke Slob, Anje Willem-sen, Alf Morten JerveSida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.2008:1:4Managing Aid Exit and Transformation: Eritrea Coun-try Case StudyTeferi Michael, Rudy Ooijen, Anneke Slob, Alf Morten JerveSida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.2008:1:5Managing Aid Exit and Transformation:Malawi Country Case StudyEsther van der Meer, Arne Tostensen, Anneke Slob, Alf MortenJerveSida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.2008:1:6Managing Aid Exit and Transformation:Botswana Country Case StudySida, Netherland’s Ministry of Foreign Affairs, Danida andNorad, 2008.Charity Kerapeletswe, Jan Isaksen, Anneke Slob, Alf MortenJerve119

dere, Annina Mattsson, Naushan Muhaimin, Adam Pain, Ric-cardo Polastro, Treena Wu.Commissioned by LRRD2 Joint Steering Committee, Sida,Norad, Danida, the Netherlands Ministry for Foreign Affairs,CIDA, BAPPENAS, Indonesia; BRR, Indonesia;Ministry for Plan Implementation, Sri Lanka, Ministry forNational Building, Sri Lanka; ISDR, Bangkok; IFRC, Bangkok;CARE International;OCHA; UNICEF, 2009.2009:3:1A ripple in development? Document review: Annotatedbibliography prepared for the joint follow-up evalua-tion of the links between relief, rehabilitation anddevelopment (LRRD) in responses to the Indian OceantsunamiJohn Cosgrave, with the assistance of: Emery Brusset, MihirBhatt, Yashwant Deshmukh, Lucia Fernandez, Yulia Immajati,Ramani Jayasundere, Annina Mattsson, Naushan Muhaimin,Riccardo PolastroCommissioned by LRRD2 Joint Steering Committee, Sida;Norad; Danida; the Netherlands Ministry for Foreign Affairs;CIDA; BAPPENAS, Indonesia; BRR, Indonesia; Ministry forPlan Implementation, Sri Lanka; Ministry for National Building,Sri Lanka; ISDR, Bangkok; IFRC, Bangkok; CARE Interna-tional; OCHA; UNICEF, 2009.2009:3:2A ripple in development? Long term perspectives onthe response to the Indian OceanTsunami: A joint follow-up evaluation of the linksbetween relief, rehabilitation and development(LRRD) – Summary ReportEmery Brusset (team leader), Mihir Bhatt, Karen Bjornestad,John Cosgrave, Anne Davies, Adrian Ferf, Yashwant Deshmukh,Joohi Haleem, Silvia Hidalgo, Yulia Immajati, Ramani Jayasun-dere, Annina Mattsson, Naushan Muhaimin, Adam Pain, Ric-cardo Polastro, Treena Wu.Commissioned by LRRD2 Joint Steering Committee, Sida;Norad; Danida; the Netherlands Ministry for Foreign Affairs;CIDA; BAPPENAS, Indonesia; BRR, Indonesia; Ministry forPlan Implementation, Sri Lanka; Ministry for National Building,Sri Lanka; ISDR,Bangkok; IFRC, Bangkok; CARE International;OCHA;UNICEF, 2009.

Commissioned by the Evaluation Departments of NorwegianAgency for Development Cooperation – NORAD, United King-dom Department for International Development – DFID, AsianDevelopment Bank – ADB, Swedish Agency for DevelopmentEvaluation – Sadev, Swedish International Development Coop-eration Agency – Sida and the Ministry of Foreign Affairs ofDenmark – Danida, 2012.2012:3Joint Evaluation of Support to Anti-Corruption EffortsNicaragua Country ReportDerek Poate, Paul Harnett, Imran Ahmed, Mignone Vega, JoseLuis VelasquezCommissioned by the Evaluation Departments of NorwegianAgency for Development Cooperation – NORAD, United King-dom Department for International Development – DFID, AsianDevelopment Bank – ADB, Swedish Agency for DevelopmentEvaluation – Sadev, Swedish International Development Coop-eration Agency – Sida and the Ministry of Foreign Affairs ofDenmark – Danida, 2012.2012:4Joint Evaluation of Support to Anti-Corruption EffortsTanzania Country ReportCharlotte Vaillant, Imran Ahmed, Deborah Mansfield, AnneBartholomew, Isaac KiwangoCommissioned by the Evaluation Departments of NorwegianAgency for Development Cooperation – NORAD, United King-dom Department for International Development – DFID, AsianDevelopment Bank – ADB, Swedish Agency for DevelopmentEvaluation – Sadev, Swedish International Development Coop-eration Agency – Sida and the Ministry of Foreign Affairs ofDenmark – Danida, 2012.2012:5Joint Evaluation of Support to Anti-Corruption EffortsVietnam Country ReportDerek Poate, Edmund Attridge, Tim McGrath, Dang NgocDung, Nguyen Thi Minh HaiCommissioned by the Evaluation Departments of NorwegianAgency for Development Cooperation – NORAD, United King-dom Department for International Development – DFID, AsianDevelopment Bank – ADB, Swedish Agency for DevelopmentEvaluation – Sadev, Swedish International Development Coop-eration Agency – Sida and the Ministry of Foreign Affairs ofDenmark – Danida, 2012.

Evaluation of Public Financial Management ReformBurkina Faso, Ghana and Malawi 2001–2010Where and why do Public Financial Management (PFM) reforms succeed? Where and how does donor supportto PFM reform contribute most effectively to results? This report summarises the findings and conclusions ofan evaluation based on studies of the PFM reforms in Burkina Faso, Ghana and Malawi. The evaluation alsodraws on a global quantitative review.It is found that results tend to be good when there is a strong commitment at both political and technicallevels, when reform designs and implementation models are well tailored to the context and when strong,government-led coordination arrangements are in place to monitor and guide reforms.Donor funding for PFM reform has been effective in countries where the context and mechanisms wereright for success, and where external funding was focused on the Government’s own reform programme.The willingness of some Governments to fund PFM reforms directly shows that external funding may not bethe deciding factor, however. Donor pressure to develop comprehensive PFM reform plans and monitoringframeworks can be a positive influence but attempts to overtly influence either the pace or the content of PFMreforms were found to be ineffective and often counter-productive. Key lessons for donor agencies are thus tofocus on countries where the right preconditions exist, to align to government programmes and, under allcircumstances, to ensure that aid works in favour of the PFM system and not against it.The evaluation has been commissioned jointly by the African Development Bank (AfDB), the Swedish Interna-tional Development Cooperation Agency (Sida) and the Danish International Development Assistance (DANIDA).

Indicator

Burkina

Faso

Ghana

malawi

Indicator

Burkina

Faso

Ghana

malawi

Data for 2009

Burkina

Faso

Ghana

malawi

BurKINA FASo

GhANA

mALAWI

PEFA Scores by Cluster (64

Indicators; maximum = 4;

minimum = 1)

Burkina Faso

2007

2010

Ghana

2006

2010

malawi

2006

2011

CoNtExt

mEChANISmS

outComES

CoNtExt

mEChANISmS

outComES

CoNtExt

mEChANISmS

outComES

CoNtExt

mEChANISmS

outComES

2008/ 2009 Afro-

barometer sur-

vey

“People are like children,

the Government should

take care of them”

“Government is like an

employee, the people

should be the bosses, who con-

trol the Government.”

Country

Date

Status

Country

Date

Status

Country

Date

Status

Countries where budget institutions

improved

Countries where budget institutions

did not improve or deteriorated

PFm reform

Countries where

Budget Institutions

Improved

Budget Institutions did not im-

prove

Intermediate

outcome

Definition

PeFA Indicators

Deliverable

Submit by

Deliverable

Submit by