Institutionalisation of EmpIAs - 6 case studies

The main objective of the review is to understand the institutional arrangements and political factors that drive EmpIAs and determine their scope; methodological frameworks and analytical instruments; roles and responsibilities in their planning and execution; and how they affect the policymaking and budget processes. Thus, the EmpIAs in the countries selected span a wide range of uses/functions from traditional evaluations of specific programmes (Australia) to the preparation of employment policies or fiscal frameworks (Cameroon and Rwanda), or the implementation of results-based budgets (Peru and Philippines) or pro-employment budgets (Korea). In all cases, however, they are implemented to influence the design of policies and programmes and the allocation of the public budget and constitute an accountability mechanism and a monitoring tool.

The country cases point to common challenges and institutional factors that affect the implementation of EmpIAs and their influence on public policy, and that would need to be taken into account by countries trying to introduce and institutionalize them.

• Political leadership and a gradual/modular implementation. In all countries, the institutionalization of EmpIAs requires a political champion and an institution that takes the lead. The two ministries that usually play this role are the ministries of finance and the ministries of labour, but there are cases where the office of the Prime Minister of that of the President are involved. In all cases, a core technical team is needed within the lead institutions to ensure the continuity of reforms efforts across government administrations. Thus, the country cases show that the introduction of EmpIAs, regardless of scope, does not happen overnight; it requires considerable time to build the necessary institutional arrangements and capabilities.
• Legal framework and fiscal incentives. Inertia and resistance to change can affect the implementation of EmpIAs. Having a dedicated legal framework institutionalizing EmpIAs and/or RBB seems to be a pre-condition for their effective deployment and development.
• Human resources. One of the bottlenecks to the introduction and diffusion of EmpIAs is not having enough in-house human resources with the necessary expertise. Initially, agencies can make up for the lack of internal capabilities by relying on individual consultants or consulting companies. But eventually, all ministries and implementing agencies involved in the employment agenda would need to have the technical capacity to manage, design, and implement EmpIAs. This implies investing in the training of existing staff and recruiting staff with expertise in project design and appraisal, cost-benefit analysis, economic modelling for ex-ante assessments, and experimental and non-experimental impact evaluations.
• Labour market information systems (LMIS) and M&E systems. The other bottleneck hindering EmpIAs are existing limitations in the M&E systems and the surveys conducted by national statistical offices. Thus, the introduction of EmpIAs requires setting up a process to gradually upgrade M&E systems. The introduction of EmpIAs might require reviewing the questionnaires of household and enterprise surveys to estimate additional KPIs (e.g., the share of informal work by age, gender, and level of education); sampling strategies to move from cross-section to rotating panels; and the frequency of the surveys from yearly to quarterly. The other important source of data are the registries and administrative systems of the social security, the tax authorities, and those of nation-wide social assistance programmes.
• Independent oversight institutions. It is necessary to have in place technical institutions to set standards for EmpIAs and evaluations in general; monitor the quality of implementation arrangements and results; and provide technical assistance. These institutions can conduct themselves the assessment of selected flagship programmes or changes in legislation, but their main role is oversight and technical assistance. In the absence of these specialized oversight institutions, it would be important to at least consider the review of instruments, methods, and results by third parties. These third parties can be other technical institutions known for having the necessary expertise; universities and think-tanks; or international organizations.
• Enforcing commitments to implement the recommendations of EmpIAs. Ultimately, the success of EmpIAs depends on whether they are able to influence budget allocations, improvements in programme design, or the adoption of new initiatives that respond to specific goals regarding employment outcomes. This could be the main challenge for all countries trying to institutionalize EmpIAs and adopt pro-employment budgets. Probably the best alternative is mandating the ministries of finance or equivalent to enforce the implementation of the recommendations from EmpIAs, either by not funding programmes that did not achieve or are unlikely to achieve the desired outcomes, or by funding conditionally on a given set of reforms. But even in these cases budgets remain subject to political pressures that might induce deviations from the desired allocation of expenditures. Ways to minimize this risk are ensuring the continuity of EmpIAs and the results of the evaluations are public and widely disseminated; and having a third party, like an employment commission, monitor and report on the results of the EmpIAs.
• National employment policies (NEPs). These are important to guide the design of new programmes or the reform of existing programmes and maximize the impact of EmpIAs. Indeed, NEPs help by setting clear targets in terms of employment outcomes that serve to guide the design of policies and programmes across line ministries. EmpIAs can also be guided by the country’s national development plans and employment policies.