APEC Project Completion Report SECTION A: Project profile Project number & title: EWG 12 2016A - APEC Fossil Fuel Subsidy Reform: Workshop on Effective Pricing Mechanisms and Mitigation Strategies Project time period: Dec 2016-Dec 2017 Date submitted: 15 Nov 2017 Committee / WG / Fora: Energy Working Group (EWG) Project Overseer Name / Organization / Economy: Elena Thomas-Kerr / U.S. Department of Energy / United States Ann Katsiak / US-ATAARI / United States SECTION B: Project report and reflection 1. Project description: In 3-4 sentences, please describe the project and its main objectives. The two-day workshop held in Jakarta from July 10-11, 2017 convened APEC officials from energy and finance ministries, and representatives from international organizations, NGOs, and academia to exchange perspectives on effective pricing mechanisms and mitigation strategies for implementing sustainable fossil fuel subsidy reforms. It built upon a 2015 FFSR capacity building workshop (under EWG 19 2015A) where economies identified additional capacity needs, particularly on addressing fuel price volatility, making reform measures resilient to price increases, identifying the most vulnerable populations and effectively targeting assistance. Through additional case studies from both APEC economies and non-APEC members, participants in this recent workshop shared lessons learned and highlighted key considerations and strategies for designing and implementing successful reforms while providing effective services to vulnerable populations. This included a comparative review of various pricing mechanisms and a discussion on the potential impacts of subsidy reform on those most affected, including gender impacts, when designing mitigation measures. Key recommendations resulting from the discussion were disseminated to APEC economies to help inform their own domestic reform efforts. 2. Meeting objectives: Describe how the project met each of its proposed objectives. Please outline any challenges you may have encountered in delivering the activity. The objectives of this workshop were to further capacity building for implementing fossil fuel subsidy reform and to develop practical solutions to common implementation challenges by sharing best practices through illustrative case studies. By exploring various pricing mechanisms and mitigation strategies, participants learned about effective options and common pitfalls to consider to
Transcript
APEC Project Completion Report
SECTION A: Project profile
Project number & title: EWG 12 2016A - APEC Fossil Fuel Subsidy Reform: Workshop on Effective Pricing Mechanisms and Mitigation Strategies
Project time period: Dec 2016-Dec 2017 Date submitted:
15 Nov 2017
Committee / WG / Fora: Energy Working Group (EWG)
Project Overseer Name / Organization / Economy:
Elena Thomas-Kerr / U.S. Department of Energy / United StatesAnn Katsiak / US-ATAARI / United States
SECTION B: Project report and reflection
1. Project description: In 3-4 sentences, please describe the project and its main objectives.
The two-day workshop held in Jakarta from July 10-11, 2017 convened APEC officials from energy and finance ministries, and representatives from international organizations, NGOs, and academia to exchange perspectives on effective pricing mechanisms and mitigation strategies for implementing sustainable fossil fuel subsidy reforms. It built upon a 2015 FFSR capacity building workshop (under EWG 19 2015A) where economies identified additional capacity needs, particularly on addressing fuel price volatility, making reform measures resilient to price increases, identifying the most vulnerable populations and effectively targeting assistance. Through additional case studies from both APEC economies and non-APEC members, participants in this recent workshop shared lessons learned and highlighted key considerations and strategies for designing and implementing successful reforms while providing effective services to vulnerable populations. This included a comparative review of various pricing mechanisms and a discussion on the potential impacts of subsidy reform on those most affected, including gender impacts, when designing mitigation measures. Key recommendations resulting from the discussion were disseminated to APEC economies to help inform their own domestic reform efforts.
2. Meeting objectives: Describe how the project met each of its proposed objectives. Please outline any challenges you may have encountered in delivering the activity.
The objectives of this workshop were to further capacity building for implementing fossil fuel subsidy reform and to develop practical solutions to common implementation challenges by sharing best practices through illustrative case studies. By exploring various pricing mechanisms and mitigation strategies, participants learned about effective options and common pitfalls to consider to help economies revise their current reform strategies or develop an action plan for those economies that are just considering subsidy reform. Five APEC economies (Indonesia, Philippines, Chinese Taipei, Thailand, and Vietnam) and two non-APEC members (Armenia and India) shared their own experiences. Additional presentations by NGOs, academia and international organizations provided more case studies as well as a more in-depth review of Indonesia’s past and current reform efforts, highlighting unsuccessful attempts and more recent more effective strategies. Among the 42 attendees, energy and finance representatives from other economies recognized similar challenges and shared their own experiences during the discussion. Based upon this frank exchange, participants identified and summarized key elements for designing successful reforms and noted that the knowledge shared during the workshop was useful for their ongoing efforts.
3. Evaluation: Describe the process undertaken to evaluate the project upon completion. (e.g. evaluation through participant surveys, peer reviews of outputs, assessments against indicators, statistics demonstrating use of outputs etc.). Provide analysis of results of evaluations conducted and where possible include information on impacts on gender. Evaluation data needs to be included as an appendix.
Input and feedback were actively encouraged throughout the workshop. At the conclusion, the PO presented key observations and recommendations raised during the 2-day discussions for review, comment and agreement by all participants. Speakers also reviewed the final workshop summary report before being circulated to APEC economies. During and after the workshop, several speakers and participants alike commented on the value of the discussions and information sharing to their respective efforts on subsidy reform. Of the 42 individuals participating in the workshop, twenty were women.
For a more formal evaluation, project overseers circulated several questionnaires to solicit participant feedback on the training structure and delivery and to assess the effectiveness of the workshop in achieving its desired outcomes. These questionnaires included: (1) Workshop Feedback Form; (2) Knowledge Gained Pre-Training Form; and (3) Knowledge Gained Post-Training. Upon completion of the workshop, POs reviewed and assessed the feedback and found that the primary workshop objective of identifying practical solutions to common implementation challenges that could help APEC economies develop or revise current reform efforts was achieved.
Participants largely held positive perceptions of the workshop, with nearly all respondents selecting ‘excellent’ (63%) or ‘good’ (33%) ratings for the overall quality of the workshop. With regard to the workshop’s delivery and format, 52% of respondents ‘strongly agreed’ and 48% of respondents ‘agreed’ that the trainers were knowledgeable about the training topics. All respondents strongly agreed or agreed that the workshop was well prepared. All respondents also strongly agreed or agreed that workshop objectives were clear and were met. In terms of participation by attendees and interaction among trainers and participants, 70% of respondents strongly agreed and 30% of respondents agreed that the workshop structure successfully encouraged such communication.
Opinions were also sought on other training features, including technical level, relevance to work and applicability to participants’ jobs. Most respondents (89%) indicated that the training level was ‘just right,’ while a small number reported that the training was either ‘too technical’ or ‘too easy.’ In terms of relevance to job responsibilities, most respondents (96%) expressed that the training was either very or somewhat relevant to their work. Moreover, all respondents confirmed that they would be able to directly apply content covered under the training ‘often’ or at least ‘occasionally’ in their work. From a broader perspective, most respondents (74%) considered the workshop topic, namely using different fiscal instruments for transitioning to market-based pricing and strategies to mitigate their impacts, to be a ‘top priority’ or ‘important priority’ for their respective economies.
To gauge whether respondents increased their capacity as a result of the workshop, two distinct hard copy questionnaires – for ‘pre-training’ and ‘post-training’ – were administered during the first and second days. Participants completed the questionnaires by rating their knowledge of specific workshop topics at the start (pre-training) and at the end (post-training) of the workshop on a scale of 0-3. Questionnaires that were less than 75% complete, or that did not have a ‘match’ – that is a corresponding completed pre- or post-training questionnaire – were discarded and excluded from the analysis. Twenty-six participants, or 65% percent of the total workshop attendees, responded to both the pre- and post- training questionnaires. Most respondents (88%) rated their capacity as having increased by 10% or more. Of 11 male respondents to the knowledge gained forms, nine (82%) indicated that their capacity increased by at least 10 percent. This compares to 14 of the 15 female respondents (93%).
4. Output indicators: Describe the main project outputs below. This may include workshops, tools, research papers, reports, recommendations, best practices, action plans..
Australia; Brunei Darussalam; Canada; Chile; Indonesia; Japan; Malaysia; New Zealand; Papua New Guinea; Peru; Peru; The Philippines; Chinese Taipei; Thailand; the United States; and Viet Nam
# speakers engaged20 18
Two speakers were unable to travel at the last minute, but their presentations were delivered by the PO
# other organizations engaged 6 6 Includes the Global Subsidies Initiative at IISD, the OECD, the IMF,
Indicators (Edit or Insert rows as needed)
# planned # actual Details or notes
The SMERU Institute, the University of Padajajaran, and University College London
# presentations distributed 20 20 20 presentations from a total of 9 sessions
Comments: The primary output was a Summary Report, which outlined the main topics, observations and key recommendations resulting from the workshop discussions. (See Appendix 4). The workshop summary report with key recommendations was distributed to the EWG members and posted on the APEC website, found here: http://publications.apec.org/publication-detail.php?pub_id=1890. All of the workshop presentations were made available online, as noted in the Summary Report: https://tinyurl.com/APEC-FFSR-Workshop .
5. Outcomes: Describe any specific medium-term changes to policy, processes or behaviour that can be attributed to result from this activity. Please include details on: What indicators were used to measure medium-term impact? (Ex indicators: type/number of policies/
regulations/processes changed, % of businesses conforming to new standards, change in sector’s commercial activity, # individual action plans developed, # agencies using resource or tools etc.)
Monitoring plans in place and proposed indicators to measure impacts, including any impacts on gender. Please summarise relevant information.
To assess medium-term impact, key indicators include revised, additional or accelerated domestic reform measures undertaken in APEC economies based on the lessons and best practices shared during the workshop, including tailored mitigation measures designed to address those most effected. Another aspect could be further analysis of gender implications of subsidy reform, as one area that needs further study as discussed during the workshop. One economy noted that they will revisit their stabilization fund to ensure that it is the most effective approach. After hearing some of the unintended consequences in implementing mitigation strategies, another economy commented that that discussion was particularly useful as they continue to pursue more effective mitigation measures. Continued information exchange and further capacity building activities will continue progress, along with new volunteers for related FFSR peer reviews. As these activities continue, they should be shared among the EWG members.
A key metric for success will be phasing out inefficient fossil fuel subsidies in the region over the long-term, in support of the APEC Leaders’ 2009 goal. The EWG tracks progress toward this goal, and the International Energy Agency provides periodic reporting on the status of fossil fuel subsidies in the region.
6. Participants/ Speakers Summary Table (compulsory for events): Must be gender-disaggregated.
Economy # male # female Total
Australia 1 1Brunei Darussalam 3 3Canada 1 1Chile 1 1People’s Republic of ChinaHong KongIndonesia 4 3 7Japan 3 3Republic of KoreaMalaysia 2 3 5MexicoNew Zealand 1 1Papua New Guinea 1 1 2Peru 1 1Philippines 1 1
Comments: What was the approach undertaken for participant nomination/selection and targeting? Please provide details. What follow-up actions are expected? How will participants/beneficiaries continue to be engaged and supported to progress this work?
The project overseers sought speakers from economies that had undertaken domestic reform efforts to share their recent experiences. Additional speakers were requested from relevant international bodies, nongovernmental organizations and academia with experience in fossil fuel subsidy work and/or fiscal instruments for transitioning to market-based pricing. All speakers were asked to provide illustrative case studies that identify key challenges and best strategies for reform. Female speakers were particularly sought, and female participants were encouraged among nominees. Project overseers requested EWG members to nominate experts with background in energy policy and reform, public subsidy policy, and/or relevant experience with pricing strategies and reform in their home economies or foreign economies. The summary report with a link to the workshop presentations were circulated to speakers, participants and EWG members to continue to inform their ongoing subsidy reform efforts.
7. Key findings: Describe 1-3 examples of key findings, challenges or success stories arising from the project (e.g. research or case studies results, policy recommendations, roadblocks to progress on an issue, impacts on gender).
Experts agreed that subsidy removal, when part of a larger policy agenda, can bring multiple gains, including more efficient energy use and improved environmental performance in addition to fiscal gains. Key recommendations included developing comprehensive reform plans with clearly defined objectives, long-term goals and transparent price adjustments; assessing how reforms could negatively affect various groups, including gender impacts, and tailor mitigation measures to address these impacts; and implementing a far-reaching communication strategy with ongoing engagement throughout the reform process to ensure transparency throughout implementation process. (A full list of Key Findings can be found in the Summary Report in Appendix 4.) Several case studies from APEC economies (Indonesia, Philippines, Chinese Taipei, Thailand, and Vietnam) and non-APEC members (Armenia and India), described key lessons learned and highlighted successful strategies for designing and implementing reforms while providing effective services to vulnerable populations. Additional examples on the impacts of firms and other case studies, including Nigeria, provided further insights. International organizations and academics described key elements for successful reforms, how to address price increases and price volatility, as well as considerations of gender implications of subsidy reform.
8. Next steps: Describe any planned follow-up steps or projects, such as workshops, post-activity evaluations, or research to assess the impact of this activity. How will the indicators from Question 5 be tracked? How will this activity inform any future APEC activities?
In addition to continued information-sharing and capacity building on pricing and mitigation strategies, participants identified other topics for follow-on workshops that could be useful, including: 1) methods for designing electricity tariff structures; 2) types of subsidies in the electricity sector; 3) producer subsidies; 4) post-reform monitoring mechanisms & case studies from developed economies where communication may differ; 5) subsidy strategies for electricity connectivity for rural and off-grid local communities; and 6) savings from subsidy expenditure to be channeled for development projects. These and other suggestions can inform future EWG capacity building activities on fossil fuel subsidy reform.
9. Feedback for the Secretariat: Do you have suggestions for more effective support by APEC fora or the Secretariat? Any assessment of consultants, experts or other stakeholders to share? The Secretariat examines feedback trends to identify ways to improve our systems.
Increased participation of finance ministry representatives along with energy ministry representatives in this workshop allowed for a well-rounded, robust discussion. It was particularly useful to engage the APEC Foreign Ministers Process to both endorse the project and consider workshop participants. Further coordination in the future will allow for continued fruitful discussions on the topic of subsidy reform. In addition, engagement of the international organizations, NGOs and academics that were invited to speak helped provide specific case studies inside and outside of the APEC region, as well as a different perspective, which allowed for additional identification of real world challenges and best practices. Future involvement of such organizations is strongly recommended.
SECTION C: Budget
Attach a detailed breakdown of the APEC- provided project budget, including: Planned costs: (using most recently approved budget figures) Actual expenditures Variance notes: An explanation of any budget line under- or over-spent by 20% or more
2 Direct labor / Contractor (clerical, secretaries, consultants)
- - - -
3 Per Diem - Speakers/Experts
$4,830.00 $2,398 -48%
Budget based on 3-day workshop in Beijing,
China. Implemented 2-day workshop in Jakarta,
Indonesia. 4 Per Diem-
Participants
$21,735 $3,597 -83%
Budget based on 3-day workshop in Beijing. Implemented 2-day
workshop in Jakarta. Only 6 travel-eligible
participants attended; budget allotted for 18.
5 Airfare - Speakers/Experts
$14,000 $4,000 -71%
Budget based on 3-day workshop in Beijing. Implemented 2-day
workshop in Jakarta. (APEC-funded airfare is
an estimate)6 Airfare -Participants
$43,000 $3,000 -93%
Budget based on 3-day workshop in Beijing. Implemented 2-day
workshop in Jakarta. Only 6 travel-eligible
participants attended; budget allotted for 18.
(APEC-funded airfare is an estimate)
7 Photocopying8 Communication
No. Item Description Budget Actual % Variance Balance Variance Notes
9 Hosting $6,000 0 -100% Hosting was self-fundedProject Total $89,565
SECTION D: Appendices
Please attach the following documentation to the report. Note that the contact list for participants/ experts/ consultants is a mandatory requirement for all Project Completion Reports.
Appendices Notes Participant contact list, including name, email address, gender,
Event Agenda Appendix 2 Reports, websites or resources created: links or soft copies Appendix 3 Post activity survey or other evaluation data (raw and/or aggregated) Appendix 4
Other information or resources FOR APEC SECRETARIAT USE ONLY APEC comments: Were APEC project guidelines followed? Could the project have been managed more effectively or easily by the PO?
APEC Guidelines were followed during project implementation.
Appendix 1. Participant/Expert Contact List
*Note : Experts are denoted as speakers under the economy breakdown.
Economy Name Gender (M/F) Institution Email
(Speaker)Other
Anna Avanesyan F Public Services Regulatory Commission
CL Israel Fierro M Ministry of Finance [email protected] ID Zulvia Dwi Kurnaini F Ministry of Finance [email protected] Fumihiro Nakayama M Japan Embassy,
The United States1 hosted an APEC Fossil Fuel Subsidy Reform Capacity Building Workshop (EWG12 2016A) in Jakarta, Indonesia from July 10-12, 2017. The two-day workshop convened APEC officials from energy and finance ministries, and representatives from international organizations, NGOs, and academia to exchange perspectives on effective pricing mechanisms and mitigation strategies for implementing sustainable fossil fuel subsidy reforms. Through case studies from both APEC economies (Indonesia, Philippines, Chinese Taipei, Thailand, and Vietnam) and non-APEC members (Armenia and India), participants shared lessons learned and highlighted successful strategies for designing and implementing reforms while providing effective services to vulnerable populations. Key recommendations resulting from the workshop are noted below.
Background
Since 2009, APEC and G20 Leaders have committed to rationalize and phase out inefficient fossil fuel subsidies that encourage wasteful consumption, recognizing that such subsidies are regressive and often fail to adequately provide essential energy services to the most vulnerable populations. Inefficient subsidies are a fiscal drain on government budgets that can hinder sustainable economic growth and limit investments in other potential public policy priority areas. Reforming these subsidies not only could reduce wasteful energy consumption but also generate savings that could be redirected to targeted alternative priorities such as providing energy access to vulnerable populations – a particularly acute need in the APEC region where over 100 million people still lack access to electricity2. APEC economies account for 60 percent of global energy demand, with significant energy demand growth expected in the coming decades. The corresponding cost of subsidies will likely also increase, underscoring the benefit of rationalizing fossil fuel subsidies. APEC
1 Hosted by the U.S. Department of Energy with support from the US-APEC Technical Assistance to Advance Regional Integration (US-ATAARI) activity2 IEA (2017). Tracking Fossil Fuel Subsidies in APEC Economies, Toward a Sustained Subsidy Reform. Paris, OECD/IEA.https://www.iea.org/publications/insights/insightpublications/TrackingFossilFuelSubsidiesinAPECEconomies.pdf
economies have made progress on subsidy reform in recent years; the International Energy Agency (IEA) estimates that APEC economies have decreased their fossil fuel consumption subsidies by 50 percent since 2011, to around USD 70 billion in 2015, both due to the decline in international prices and government policy reforms. However, the potential for international oil price increases in the future may test the long-term durability of reforms.
Common Challenges
There are a variety of common challenges to implementing reforms, including a general lack of understanding about the size and drawbacks of fossil fuel subsidies and their impacts on public spending, including for poverty alleviation. The public may also distrust the government’s ability to implement such reforms in a way that minimizes the impact, particularly if there are weak macroeconomic conditions (e.g., high inflation and currency depreciation). Governments also face opposition from influential groups who benefit the most from subsidies and may seek to block reforms. Below are three primary challenges highlighted in the workshop: establishing effective pricing mechanisms, ensuring successful mitigation strategies and implementing a far-reaching communication strategy to gain support for reforms.
Effective Pricing Mechanisms
Participants discussed approaches to effectively transition toward market-based pricing, prepare for potential significant fuel price increases, and mitigate impacts on the most vulnerable populations. Experts reviewed various fuel pricing mechanisms to transition toward market-based pricing, assessed their fiscal sustainability and the degree to which they minimize price fluctuations and fiscal volatility.
Experts concluded that gradual yet frequent price adjustments through an automatic pricing mechanism based on international prices have often been most successful. A transparent adjustment formula that is widely published, and the establishment of independent agencies to administer the adjustments and audit their implementation have also helped provide predictability and depoliticize reforms, thus enhancing their durability. Participants also cited the importance of establishing contingency plans for significant fuel price increases (differing by fuel type), and simultaneously investing in areas such as energy efficiency, clean energy subsidy swaps and investments in social programs, which can help offset the effects of price increases.
Managing Impacts of Reform
Several presenters stressed that effective reforms require tailored mitigation strategies for the most vulnerable. They noted the importance of clearly identifying those most affected and of understanding the impacts, including gender impacts of reform. However, identifying “vulnerable” groups based solely on income or strictly on energy consumption can miss key considerations; different income groups rely on different energy types and various factors can determine vulnerability and exposure to income shocks. Contributors recommended that policy-makers and planners consider income levels, location, energy access and other factors in identifying vulnerable populations and designing mitigation measures. Linking subsidies to established social assistance programs may help better target those in need. Cash transfers are often the preferred approach for mitigation. However, where cash transfers are not feasible, subsidies can be implemented through benefits such as education, health care and public works.
Mitigation measures may have unintended consequences. For example, some governments have found that certain delivery mechanisms can make it difficult for rural or remote populations to obtain the benefit. Another unintended consequence can be environmental degradation (e.g., deforestation, dirtier fuels) if a primary energy source becomes unaffordable and people must turn to other less sustainable energy sources. Some presenters recommended that governments also consider populations near the poverty line that could fall into poverty if reforms do not include appropriate compensation. In devising mitigation efforts, some suggested the importance of monitoring and reassessing programs during implementation to ensure the measures are properly targeted. For example, one case study described how fuel cylinders sold at a subsidized price led to black market trade. The government changed the program to sell the fuel cylinder at market prices and provided direct transfers of the subsidy to those in need, thereby targeting the subsidies and minimizing diversion.
Reform measures can also put pressure on firms, particularly small- and micro-firms that can be forced out of the market. Firms can be affected directly through increased energy input costs, or indirectly as rising energy costs are passed along the supply chain to non-energy inputs (e.g., materials, capital and labor). To address this impact, governments can provide short-term support to small- and micro-firms and/or energy-intensive sectors to allow time for adjustment with the goal of improving energy efficiency and investments in energy-saving technologies. Generally, transparency and policy certainty during the reform process will also help firms adjust.
Communication Strategy
Participants agreed that effective reform measures require extensive coordination across the government, industry, media, NGOs, academia, other influential groups (e.g., unions, ethnic or religious leaders), and those most impacted by reforms. Since reforms are politically sensitive and can have economic and social impacts, reform implementation requires sustained public support with clear and sufficient information. Effective messaging is critical to justify the need for reform and communicate complex subsidy policy and pricing to provide sufficient clarity.
Experts advised development of a far-reaching communication strategy and pursuing transparency and inclusivity throughout the process, including public discussions to address concerns. Effective strategies include knowing your audience and how they will receive the message and conveying the message through the most appropriate medium. Social media will play a significant role in the role out of reforms, particularly for some segments of the population. Presenters recommended that governments track and communicate progress during reform implementation by posting information online and through public outreach.
Some participants suggested that governments can also promote transparency by participating in international peer reviews and employing widely available tools and resources such as the OECD Inventory of Support Measures. One expert noted that the peer review process itself can be useful as it requires interagency coordination and helps an economy reconsider the rationale of subsidies and whether they are still relevant and effective.
WORKSHOP RECOMMENDATIONS
Experts agreed that subsidy removal, when part of a larger policy agenda, can bring multiple gains, including more efficient energy use and improved environmental performance in addition to fiscal gains. Key recommendations from the discussions are noted below.
Planning/Pricing:
Develop comprehensive reform plan with clearly defined objectives & long-term goalso Include contingency plans for significant price increases (by fuel type)
Implement gradual price increases with transparency around adjustmentso Provides predictability, and shows reforms are part of a broader strategy
Improve efficiency of state owned enterprises and strengthen governance to eliminate inefficiencies
Provide temporary support for energy intensive sectors and/or small- and micro- firms that could otherwise be driven out of the market
Depoliticize fuel pricing o Automatic pricing mechanisms with price smoothing can be most effectiveo Establish independent body to administer adjustments and/or conduct audits
Implement parallel activities such as investments in energy saving technologies and energy efficiency improvements to reduce wasteful consumption
Mitigation measures:
• Understand how reforms would affect various groups, including gender impacts o Consider income, energy use, energy supply, location and other factors to
identify “vulnerable” groups• Assess adequacy of social safety net and utilize existing social programs where possible• Target subsidies to those most affected and through effective delivery mechanisms• Reassess mitigation measures along the way to address unintended consequences• Consider impacts on small-/micro-firms and provide short-term assistance as needed• Investing in social services (e.g. health, education, transport, clean energy) can help
mitigate impacts
Communication/Transparency:
• Implement far-reaching communication strategy with ongoing engagement o Engage policymakers, critical interest groups, industry, civil society, vulnerable
populations, key opposition groups• Provide clear message on the need for reform and how savings will be used• Consider cultural, social, generational aspects in messaging and most effective media • Deliver extensive information and transparency throughout implementation process• Take advantage of previous experience, other case studies, and international tools to
build the case for reform, benefit from lessons learned and implement best practices.
Appendix 4. Post Activity Survey / Evaluation Data
Monitoring and Evaluation DataAttendee ProfileAs noted above, the workshop was designed to provide capacity building and facilitate a dialogue on common challenges and key elements to success for fossil fuel subsidy reform in the region. The workshop drew a total of 42 attendees from a range of public and private sector stakeholders including high level technical officials from energy and finance ministries, as well as experts from academia and international organizations (see participant profile in Figure 1).
Figure 1: Profile of Attendees (N=42)
Attendee SatisfactionData presented in this section were collected anonymously through the paper-based workshop feedback forms administered at the end of the workshop. Twenty-seven participants, or 68% of the total workshop attendees, responded to the questionnaire.
Responses indicated that participants largely held positive perceptions of the workshop, with nearly all respondents selecting ‘excellent’ (63%) or ‘good’ (33%) ratings for the overall quality of the workshop (see Figure 2).
Figure 2: Workshop Overall Rating and Feedback
With regard to participant views on the workshop’s delivery and format, 52% of respondents strongly agreed and 48% of respondents agreed that the trainers were knowledgeable about the training topics. All respondents strongly agreed or agreed that the workshop was well prepared. All respondents also strongly agreed or agreed that workshop objectives were clear and were met. In terms of participation by attendees and interaction among trainers and participants, 70% of respondents strongly agreed and 30% of respondents agreed that the workshop structure successfully encouraged such communication. Figure 3 below presents further detail on the respondents’ perceptions as to the workshop’s delivery and format.
Figure 3: Specific Aspects of the Workshop
Respondents were also asked to provide their feedback on the usefulness of the various sessions of the workshop. Detailed results are shown in Figure 4. Most topics covered in the training sessions were rated highly with respect to their usefulness. In particular, respondents identified the discussion on effective pricing mechanisms as especially useful.
n=27
n=27
Figure 4: Usefulness of Training Sessions
Workshop participants’ opinions were also sought on other training features, including technical level, relevance to work and applicability to participants’ jobs. Most respondents (89%) indicated that the training level was ‘just right,’ while a small number reported that the training was either ‘too technical’ or ‘too easy.’ In terms of relevance to job responsibilities, most respondents (96%) expressed that the training was either very or somewhat relevant to their work. Moreover, all respondents confirmed that they would be able to directly apply content covered under the training ‘often’ or at least ‘occasionally’ in their work. From a broader perspective, most respondents (74%) considered the workshop topic, namely using different fiscal instruments for transitioning to market-based pricing and strategies to mitigate their impacts, to be a ‘top priority’ or ‘important priority’ for their respective economies.
Figure 5: Additional Training Features
Knowledge GainedIn order to gauge whether respondents increased their capacity as a result of the workshop, two distinct hard copy questionnaires – for ‘pre-training’ and ‘post-training’ – were administered during the first and second days. Participants completed the questionnaires by rating their knowledge of specific workshop topics at the start (pre-training) and at end (post-training) of the workshop.3 Questionnaires that were less than 75% complete, or that did not have a ‘match’ – that is a corresponding completed pre- or post-training questionnaire – were discarded and excluded from the analysis.4 Twenty-six participants, or 65% percent of the total workshop attendees, responded to both the pre- and post- training questionnaires. The figure below summarizes responses to the baseline and endline questionnaires.
3 Closed-ended questionnaires are circulated pre-training and post-training at workshops asking participants to evaluate, on a five-point scale, their level of knowledge on the subject matter before (baseline) and after (endline) the training.4 Due to a lack of matches, there were 5 discards for Form #3a Knowledge Gained Pre-Training; and 1 discards for Form #3b Knowledge Gained Post-Training.
Figure 6: Respondents’ Knowledge (Baseline vs. Endline)
As to self-reported knowledge of training subject matter prior to the workshop, a majority of respondents rated their knowledge as ‘none’ or ‘limited’ across categories. More than half of respondents (58%) rated their understanding of mitigation strategies and options for managing the impacts of reform as ‘none’ or ‘limited’ at the outset. Over half of respondents (58%) also indicated they had ‘no’ or ‘limited’ understanding of engaging stakeholders, tracking implementation, and conveying progress on FFSR to the public. While the audience on average self-reported at least some familiarity with the subject matter at baseline, participants clearly could benefit from discussion of common challenges and key elements to success for fossil fuel subsidy reform in the APEC region.
Responses to the endline questionnaire clearly illustrated a gain in immediate knowledge and understanding following the workshop (see Figure 7). Whereas a majority of respondents had rated their understanding of mitigation strategies and options for managing the impacts of reform as ‘none’ or ‘limited’ at baseline, 42% of respondents rated their knowledge as ‘high’ at the workshop conclusion. Respondents rating their knowledge of engaging stakeholders, tracking implementation, and conveying progress on FFSR to the public as ‘medium’ or ‘high’ increased by 57% (from 43% to 100%). Respondents assessing their knowledge on various pricing mechanisms for FFSR and their respective effects as ‘medium’ or ‘high’ also increased by 54% (from 46% to 100%). Of note, all respondents indicated they had a ‘medium’ or ‘high’ understanding across all categories at endline.
Figure 7: Average Respondents’ Knowledge (Baseline v. Endline)
As illustrated in Figure 7, workshop participants as a group did on average self-report considerable knowledge gain across the training topics. With respect to by how much the respondents on average
n=26
n=26
increased their capacity, Figure 8 illustrates the percentage of respondents stating their capacity has increased by at least 10 percent. Most respondents (88%) rated their capacity as having increased by 10% or more. Of 11 male respondents to the knowledge gained forms, nine (82%) indicated that their capacity increased by at least 10 percent (see Figure 9). This compares to 14 of the 15 female respondents (93%).
Figure 8: Self-Reported Knowledge Gain (≥10% Knowledge Gain)
Figure 9: Self-Reported Knowledge Gain (Sex Disaggregated)
82% 93%of male respondents reported their capacity had increased by 10% or more.
vs.
n=26
n=11 n=15
of female respondents, reported their capacity had increased by 10% or more.
