Find Apraise on...

Facebook icon LinkedIn icon Twitter icon Flickr icon YouTube icon RSS icon

Apraise News


Apraise newsletter

Stay informed on our latest news!

APRAISE Case Studies

Find more here!


Access the APRAISE Share here.

You are here



The key objective of the APRAISE 3E method is to enhance policymakers’ capability to design, understand, evaluate, and implement policy instruments aimed at achieving environmental targets within the context of a broader policy and stakeholder system. Ideally, policymakers base their decisions on the best knowledge available to them at a given point in time concerning the policy environment, and the mechanisms through which policy instruments are expected to lead to the desired goals, including assumptions concerning the ways in which stakeholders respond to these instruments. These ‘upfront’ expectations, which can be based on available theory and experiences, possibly supported by models, are referred to in APRAISE as the efficacy of policy instruments. After implementation, the policy (and the associated instruments) can be evaluated by examining its actual effect: the effectiveness and efficiency of a policy.


Policymaking is complex and can be an ad hoc process. The political, economic and social policy context is difficult to control and changes in these contextual factors may influence the effectiveness and efficiency of a policy instrument (see Figure 1). Therefore, policy design and implementation (such as assumed governance procedures) may deviate from what was expected. Finally, stakeholders targeted by the policy instruments are often targeted simultaneously by several other environmental or other policies and policy instruments, which make their behaviours difficult to predict.








This diagram illustrates how the system context for policy instruments, policy implementation aspects and policy interaction through stakeholder behaviour can influence the effect or impact of policy instruments. This could lead to an outcome whereby the actual impact or effect (effectiveness) of policy instruments differs from anticipated, theory-based impacts/effects (efficacy). Please note that policymaking in reality is usually not as linear as shown in the diagram..

Figure 1: Policy making in reality – policy impact/effect influenced by contextual, implementation and policy interaction factors


In virtually all policy instrument design processes, various assumptions are made about the system context factors (context factors in the nation state, sector or policy area) that could negatively or positively affect the operation of policy instruments during the implementation stage. In a few cases, such assumptions are sufficiently understood and made explicit (e.g. expected economic growth or anticipated collaboration between stakeholders) during the design of a policy instrument. However, the assumptions often remain implicit or are unknown and are therefore not properly accounted for in the policy instrument design stage. Therefore, the detailed and specific design of an individual policy instrument includes to a certain extent a reflection on how well the context of an individual policy instrument has been taken into account before the implementation stage. As a result, the achieved policy impact or effect (effectiveness) may, in practice, differ considerably from what policy makers anticipated in the light of their efficacy knowledge.

Deviations between efficacy and effectiveness can also influence the efficiency of policies. In an optimal situation, the intended policy effect is achieved with the lowest costs possible. In reality, however, efficiency may be lower because the goal has been achieved at higher than anticipated costs, or it has not been achieved but costs have been incurred as expected. Opposite conclusions of efficiency can be drawn if the outcomes are better than anticipated. In line with the discussion on efficacy and effectiveness, the extent to which policymakers are able to design an efficient policy partly depends on their ability to correctly anticipate the impact of the context, policy implementation aspects, and stakeholder responses.

One of the key lessons from APRAISE is that a better understanding of contextual, implementation and policy interaction aspects enables policymakers to design more robust policy instruments, which implementation and operation can be adapted to (foreseen) changes in the circumstances (context). The APRAISE 3E method, in combination with the case studies in which it has been applied, offers a key tool that helps to inform policymakers about these aspects and to enhance environmental policymaking. Figure 2illustrates this learning process.

learning effects based on 3E.png

Figure 2. Illustration of how the APRAISE 3E method helps policymakers make better informed assumptions about the efficacy of policy instruments


How does the APRAISE 3E method work?


The APRAISE 3E method helps policymakers to more systematically assess the anticipated effect(s) of a policy (instrument) during its design stage, by making better-informed assumptions about their contextual, implementation and stakeholder behaviour aspects (including possible interactions with other policy instruments). The method, therefore, helps to close the gap between expected/intended and achieved policy effects and impacts. At the same time, the method is applicable in any member state and lessons from these applications can be relevant for multiple policymaking levels.

The APRAISE 3E method adopts a systems approach to examine an event or a system in a holistic manner by emphasising the relationships and interactions between the system’s elements (i.e. the actors involved and the institutions governing their interrelationships). The APRAISE 3E method assesses the following groups of factors:


System context factors, such as:

·      Environmental factors: The original perception of the extent of an environmental stress/pressure that motivated the design of a policy and the choice of policy instruments may change over time, thereby possibly shaping the ability of the policy or instruments to achieve their targets.

·      Economic factors: An environmental policy may change the structure of an economy, supporting some sectors and penalising others. At the same time, economic developments (e.g. GDP growth, energy prices, trade conflicts, and climate policies) and institutions involved in policy implementation can influence the effect of an environmental policy.

·      Social factors: Social factors include habits, customs and social attitudes. They can either oppose the implementation of a policy and related instrument (e.g. where a target is not fully accepted by society) or support its implementation (e.g. where a policy increases environmental awareness and thereby support the policy).

·      Technological factors: When designing a policy, a certain technological capacity may be assumed as a precondition for achieving the policy target(s). During policy implementation, additional technological inventions and innovations may take place (either autonomously or as an unintended impact of the policy) and help achieve the policy target(s).


Policy implementation factors, such as:

·      Political & social acceptance: This refers to the political and social response during policy formulation, implementation and evaluation. Key design elements of a policy instrument can generate or ease resistance of target groups in accepting a policy.

·      Policy consistency with wider environmental and sustainability goals: This reflects the extent to which an environmental policy objective and a policy instrument are compatible with other policy priorities, including other sustainable development goals.

·      Policy coherence: Policy coherence within APRAISE is associated with the public process, i.e. harmonisation, coordination and cooperation procedures across government departments and agencies. Coherence denotes the degree of alignment of incentives to promote sustainability, both vertically across levels of government and horizontally across different actors and issues within a given level of governance. Closely linked to policy coherence is the practical feasibility of implementation (or enforcement) of a policy. This feasibility relates to the applicability of the policy instrument, considering the national institutions (including infrastructure and human resources) and legislative framework.


Policy and stakeholder interactions are assessed in APRAISE at two levels:


Interactions at the policy level occur when policy targets or policy instrument design features may influence the operation or outcome of another policy or policy instrument. In the design phase, negative interactions should be avoided or at least kept within acceptable margins. Interactions are examined by assessing design features of a policy instrument such as type, activity scope, timing, etc.


Interactions at the stakeholder level relate to the direct and indirect impacts of policy instruments on the behaviour of stakeholders. The APRAISE 3E method uses a ‘system mapping’ tool to analyse how stakeholders compete or collaborate with each other and how this may influence their response to policy instruments. The tool also explores enabling aspects to help stakeholders modify their behaviour in response to a policy instrument.


An overview of the eight steps to be completed under the APRAISE 3E method is shown in Figure 3. After identifying (task 1) and characterising (task 2) policy instruments, the effectiveness and efficiency of these instruments are analysed and compared with the anticipated effects based on efficacy knowledge (task 3). Insights from analysis of the policy system context, policy transposition and implementation process and possible policy interactions (tasks 4-7) can then be used to improve policymakers’ knowledge of the efficacy of policy instruments (task 8).



Figure 3. Overview of steps in APRAISE 3E method


A short guide to the APRAISE 3E Methodology is available here.

Impact Stakeholders’ Questionnaire

Policy Brief

The 2nd APRAISE Policy Brief provides a thorough analysis of the role of sustainable development indicators for evaluating the effectiveness, efficiency and efficacy of policy instruments. It also offers an assessment on their usefulness in answering research questions related to the issues addressed by APRAISE. The 2nd Policy Brief is available here.

Policy Brief

You can find the 1st Apraise Brief here.

APRAISE Synthesis Document

The APRAISE Synthesis Document presenting the results of the APRAISE 3 years of operation has been released! The APRAISE Synthesis Document is available here:

APRAISE Synthesis Document