Success case method

The Success Case Method (SCM) involves identifying the most and least successful cases in a program and examining them in detail.

This approach was developed by Robert Brinkerhoff to assess the impact of organisational interventions, such as training and coaching, though the use of SCM is not limited to this context. It is a useful approach to document stories of impact and to develop an understanding of the factors that enhance or impede impact.

The Success Case Method deliberately looks at the most, and least, successful participants of a program. The purpose is not to examine the average performance - rather, by identifying and examining the extreme cases, it asks: 'When the program works, how well does it work? What is working, and what is not?'.

An SCM study can be used to answer any, or all, of the following questions:

• What is really happening?
• What results, if any, is the program helping to produce?
• What is the value of the results?
• How could the initiative be improved?

Brinkerhoff, 2003, p. 6-7

The immediate results of conducting a Success Case Method study include documented stories of impact that can be disseminated to stakeholders, and a better knowledge of factors that enhance or impede business impact.

There are five key steps in the SCM method (Brinkerhoff, 2003, p.29):

1. Focusing and planning a Success Case study

2. Creating an “impact model” that defines what success should look like

3. Designing and implementing a survey to search for best and worst cases

4. Interviewing and documenting success cases

5. Communicating findings, conclusions, and recommendations

Example

In a guest blog on BetterEvaluation, Liz McGuinness discusses the implementation of the Success Case Method as part of the USAID Complexity-Aware M&E Trials. In this example, the Success Case Method was adapted and applied to a multi-country capacity building project funded by USAID. The program provided training and support to a small number of professionals who were placed in new positions within government departments. The objectives for using this method were to support the donor and implementer to adaptively manage the project and to discover the, as yet unknown, development pathways from the project activities to the desired outcomes. The trial surfaced several key lessons that are applicable for those considering using this approach. You can read Liz's reflections on the challenges encountered and the results here: Lessons from a trial of the Success Case Method

Advice

The following is taken from Liz McGuinness' guest blog: Lessons from a trial of the Success Case Method.

Advice for CHOOSING this method

• When you know the long-term objectives of your program and you have identified your program activities but you do not know the causal pathway from activities to impact.
• When you can identify both the program participant and at least one individual who oversees their work and can vouch for both the participant’s actions and the outcomes of these actions.
• When the M&E team can be given authority to directly communicate with, and collect data from both the participant and supervisor subjects.

Advice for USING this method

• The M&E team who are to conduct SCM evaluations should include at least one member who can design, administer and analyze surveys and at least one member who has strong qualitative data analysis skills.
• Ensure that the evaluation is included in the project design, workplan and budget so that does not become added work for the staff.
• Use an online survey platform (ex. SurveyMonkey, Survey Gizmo, etc) to make data collection and consolidation more efficient. Find out in advance what kind of internet access your respondents have before deciding on the survey approach.
• Create an Excel database to consolidate the survey data, to analyze the closed questions, and to automatically identify success and non-success cases.
• Create templates for analyzing the open questions to serve as guides.
• If the Success Case Method results are to be used to support adaptive management of your project, ensure that the schedules of the evaluation and of management decision-making are in sync. That is, make sure that your results will be reported out sufficiently before decisions are made about annual workplans, to be taken into account.

Resources

Overview

This book by Brinkerhoff runs through the entire process of conducting a Success Case Method Study and includes a number of examples of use

Identification and reporting of critical success factors and program impact in an efficient, yet comprehensive manner is an inherent difficulty facing many evaluators of large-scale evaluations. This paper details how two evaluators encountered such problems in the initial review of a large-scale initiative and then successfully addressed these issues through the application of the Success Case Method in a subsequent evaluation of the same program. (Catherine Bell and Diane McDonald, Paper at Australasian Evaluation Society International Conference, Darwin, 2006).

A brief, downloadable e-book on using the method. It includes suggestions for pairing the approach with other methods.

This post discusses the author's use of SCM as compared to the Kirkpatrick evaluation model for evaluating learning.

Examples

This method note demonstrates how SCM was modified and extended to a social service context, in which the program evaluated was aimed at reducing chronic homelessness and unemployment.

This study employed Success Case Method to understand how 3 performance improvement CME activities contributed to implementation of tobacco cessation practice guidelines in 9 outpatient practices.

In this guest blog, Liz McGuinness shares her lessons from trialling the Success Case Method as part of the USAID-funded Complexity-Aware M&E Trials.

Brinkerhoff, R. O. (2003). The success case method: find out quickly what’s working and what’s not. San Francisco: Berrett-Koehler