Measuring and evaluating impacts

Continuously monitoring the impact of RSPO’s work is essential for the development of our standards. It also helps us assess the ways in which our work contributes to our overall vision of transforming markets to make sustainable palm oil ‘the norm’. In order to achieve this, the RSPO Impacts and Evaluation team has strengthened their commitment to developing, implementing, and improving its ‘Monitoring and Evaluation’ (M&E) system.

 

How we measure impacts

Our team use a three-level structure for collecting and assessing data. The first level is data that is collected yearly from all certified units, new planting procedures and RSPO’s Annual Communication of Progress (ACOP). The second level focuses on monitoring data from a sample of certified units, while the third level centres around data collection from in-depth independent and commissioned studies. To date, RSPO has collected a large amount of level one data and are making significant progress on level’s two and three. An in-depth, five-year, independent research project; Socially and Environmentally Sustainable Oil Palm Research or ‘SEnSOR’ has already been established and the first phase is complete. 

 

Why a Monitoring & Evaluation System?


In addition to helping us capture more detailed information of the effectiveness of our work, an efficient M&E system allows us to identify any critical changes within the palm oil supply chain and adapt accordingly. By implementing this we have more ownership and accountability, and can evaluate broader or longer-term results.


The RSPO Monitoring & Evaluation system uses an Impact Monitoring Framework to identify the intended outcome and measure the progress of each activity. This structure ensures that each activity delivers value across all levels of the palm oil supply chain, based on three key elements: environmental, social, and economic benefits (read more on the M&E system documents here).

 

RSPO is also working to strengthen and improve its M&E system by developing a Theory of Change (ToC) and identifying performance indicators which is scheduled for completion by end of 2017. Below is a diagram of the process and flow with a timeline for the ToC development project. 

 

 

Listen & Learn

  • Review of key organisational and project related documentation
  • Calls with key Secretariat team members
  • Interviews with key stakeholders
  • High level mapping of RSPO system elements, identifying components of their underlying theory of change/impacts elements
  • Review of other systems
  • The Listen and Learn Report can be downloaded in here.

Draft RSPO Straw-man Theory of Change (ToC)

  • Draft Straw-man Theory of Change and draft narrative that will be used for workshops.

Theory of Change (ToC) Workshops

  • To identify the already existing elements within the existing RSPO (certification) processes that support change
  • Validate what is correct on the Straw-man Theory of Change and highlight what is correct
  • Identify missing content, areas for further exploration
  • Strengthen causal pathways, recognise interdependencies, areas of impacts vs. influence and contextual factors, improve metrics and KPIs
  • Finalisation & Socialisation
  • To produce a high-level communications (external facing) Theory of Change with short narrative

Draft M&E System Components

  • Defining indicators using what exists where possible, and identifying what is needed in terms of SMART criteria (Specific, Measurable, Attainable and action-oriented, Relevant and Time-bound), ISEAL Common Core Indicators, and relevant benchmark frameworks
  • Draft M&E plan that covers both the content (e.g. Theory of Change and indicators along the causal chains) and implementation needs (e.g. data system, policies)

Facilitate Feedback & Finalisation of M&E System

  • Final M&E Plan
  • Prioritised indicators organised by levels including definitions, and data collection protocols (methodologies, timing, etc.)

Debrief & Insights for the Future

  • An overview of the process and results will be documented.

CONTACT US

If you have questions or comments about the RSPO's Monitoring and Evaluation System, please submit them to Elisa Chong.

 

 

RSPO IMPACT REPORTS

Discover more about the RSPO’s impacts in recent years by downloading the Impact Reports below:



Impact Report - 2016
English (16 MB)
Impact Report - 2015
English (6 MB)
Impact Report - 2014
English (6 MB)

 

M&E AND THEORY OF CHANGE DOCUMENTS



Listen and Learn Report - 2017
English (685 KB)
M&E Public System Report - 2017
English (808 KB)
Impacts Code Report - 2016
English (541 KB)
Impacts System Report - 2013
English (1 MB)

 

RSPO COMMISSIONED RESEARCH STUDIES



Impacts of RSPO Certification on Economic on Smallholders in Jambi, Indonesia - Bahasa - 2015
Indonesian (2 MB)
The Impacts of RSPO on the Livelihood of Smallholders – English – 2015
English (9 MB)
Correlating Economic and Financial Viability with Sustainability - 2015
English (1 MB)

«