Most Significant Change Methodology as Part of M&E

Dear Partners:

This month’s eUpdate is dedicated to a specific method of documentation which can be applied as part of monitoring and evaluation processes. For those of you who have undergone more than one cycle of the IDF, you have heard of this before, as it is a process we apply to document impact derived from the OS process overtime. I am referring to the Most Significant Change (MSC) approach. The MSC is an innovative and unconventional approach to documenting impact and organizational learning. It is a qualitative approach that involves generating and analyzing personal accounts of change and deciding which of these accounts is the most significant – and why.

The are three basic steps in using MSC:

1. Deciding the types of stories that should be collected (stories about what – for example, about practice change or health outcomes or empowerment)
2. Collecting the stories and determining which stories are the most significant
3. Sharing the stories and discussion of values with stakeholders and contributors so that learning happens about what is valued.

The MSC is not just about collecting and reporting stories but about having processes to learn from these stories – in particular, to learn about the similarities and differences in what different groups and individuals value.

It provides some information about impact and unintended impact but is primarily about clarifying the values held by different stakeholders. By itself it is not sufficient for impact evaluation as it does not provide information about the usual experience but about the extremes. If you imagine a normal distribution of outcomes for individuals then the stories often come from the extremity of positive change.  It can be useful to explicitly add a process to generate and collect stories from the extremity of little or negative change.

The MSC can be very helpful in explaining HOW change comes about (processes and causal mechanisms) and WHEN (in what situations and contexts). It can therefore be useful to support the development of program theory (theory of change, logic models). 

But how to do you know when the MSC is the right approach to use?

Here are some helpful “Tips” and “Traps” to consider when determining methods:

• MSC is particularly useful when you need different stakeholders to understand the different values that other stakeholders have in terms of “what success looks like” – criteria and standards for outcomes, processes and the distribution of costs and benefits.
• MSC works best in combination with other options for gathering, analyzing and reporting data. It doesn’t provide comprehensive information about the impacts produced by an intervention. 
• Ensure the stories are not high jacked for other purposes such as for promotional material. Data can only be used for the original stated purpose, which in this case is evaluation unless other uses have been negotiated and agreed to at the time.
• MSC is not a quick option.  It takes time and an appropriate project infrastructure to generate understanding and value clarification (identifying what people think is important). The full MSC process involves analysis of stories and sharing with both contributors and stakeholders, which requires a program with several structures in it (for example, local, regional and national project structures) and it needs to be repeated through several cycles.
• There is scope to be innovative in this option. Your project may not have a hierarchical structure so there may be other ways of forming groups around which the stories can be discussed and the values identified.
• It can be challenging to get engagement of the different groups involved in the process and to maintain their interest.  Don’t have too many cycles of review. 
• Other Skills Necessary: Good facilitation skills are important along with the ability to identify priorities.

Want to learn more?

Even better, want to apply the MSC as part of your qualitative documentation of impact? Here are some helpful resources for you to practice applying the MSC in your own work. 

The ‘Most Significant Change’ Technique – A Guide to Its Use

This document is considered essential reading for anyone seeking to use the MSC approach. Written by the two developers of MSC (Rick Davies and Jess Dart), it provides an overview, clear guidance on MSC in 10 steps, and advice on troubleshooting and incorporating MSC in an M & E system. This 9 chapter comprehensive manual takes you in depth through processes applying the MSC considering your M&E frameworks which guide your work. 

• To access the English version of the manual click here.
• To access the French version of the manual click here.

Strategy Development: Most Significant Change (MSC):

This is a summarized guide to the Most Significant Change approach was written by the Overseas Development Institute (ODI). It first appeared in the ODI Toolkit, Tools for Knowledge and Learning: A Guide for Development and Humanitarian Organizations. It provides an overview, a detailed description of the process, and an example of the technique in action.

Click here to download the PFD.

Example: Most Significant Change (MSC) used for NGO self-evaluation process:

This document from Capacity.org outlines the process and findings of an NGO self-evaluation process which used the MSC approach to evaluate the organizational development services offered by the group. The self-evaluation process was designed with the explicit purpose of ’enhancing CABUNGO’s learning in order to improve performance’. The evaluation team decided to use this opportunity to trial the use of the story-based Most Significant Change (MSC) methodology to evaluate the organizational development services provided by CABUNGO. This document serves as an example of what a finished product looks like applying the MSC approach to documenting impact and organizational learning. 

Click here to download the PFD.

Online video of an MSC Training workshop:

The below is a 4 part video stream of an MSC training conducted by International Development Research Center (IDRC). These videos provide an alternative to those who would prefer a different method of learning than reading the manuals alone. This video series provides the “how-to” with some overview of the MSC’s application by IDRC and their experiences using it to document stories of impact. 

• Part 1
• Part 2
• Part 3
• Part 4

More to come!