December 2003
| Periodic evaluation during on-going work can help shape the course of future work. Such formative evaluation is particularly relevant to the evaluation of conflict management and resolution work, particularly in intractable conflicts, where we so often face shifting conflict dynamics and shifts in conflict resolution emphasis over the long-term. |
Formative evaluation is particularly useful for evaluating long-term conflict resolution work in intractable conflicts. In these conflicts, interconnected disputes, with multiple causes and systemic impact, the dynamics, participants, and even issues may change over time, and thus conflict resolution activities need to refocus on the new realities as they emerge. In fact, it may be useful to conduct repeated formative evaluations periodically throughout the life-cycle of a long-term conflict resolution process.
What is Formative Evaluation?
At its most basic, formative evaluation is an assessment of efforts prior to their completion for the purpose of improving the efforts. It is a technique that has become well developed in the education and training evaluation literature.[1]
For conflict resolution efforts addressing intractable conflicts, appropriate formative evaluation can be a powerful part of an intervention strategy. By assessing efforts, and adjusting strategies in response to the assessment, conflict resolution practitioners can better respond to the changes in conflict dynamics that occur in intractable conflicts. Conflict resolution practitioners can thus recalibrate their efforts to better respond to current conflict dynamics, and make other mid-course corrections as necessary. In addition, by monitoring implementation through a formative evaluation, conflict resolution practitioners can also recognize opportunities to fine tune or even redefine their strategies. Finally, by involving the parties in the formative evaluation effort, conflict resolution practitioners can develop additional commitment on the part of participants and funders to conflict resolution. For conflict resolution work, formative evaluation offers an opportunity to keep everyone on board, and either keep on track or determine when it is necessary to switch tracks.
Robert Stakes is quoted as saying, "When the cook tastes the soup, that's formative. When the guests taste the soup, that's summative."[2] Formative evaluation can take many forms. It is most often conducted by the conflict resolution practitioner (the cook in Stakes' metaphor), but many involve all the stakeholders. (For more discussion of this multi-stakeholder approach, see Action Evaluation.)
Key Roles for Formative Evaluation
Formative evaluation encourages a process of reflective practice. More specifically, formative evaluation can strengthen conflict management systems in many ways. These include:
- Rapid feedback. Primarily, formative evaluation provides rapid feedback on the efficacy of conflict management and resolution work. While a project in is progress, a formative evaluation process provides feedback on how the work is going.
- Documentation. A formative evaluation process can document how conflict management and resolution work is proceeding, what techniques are used, what problems encountered, and what impacts are made in early and middle stages of work. Such documentation may be useful
- Planning. Formative evaluations assist with planning and allows for revision of or recommitment to plans. Formative evaluation involves a comparison of program implementation with program plans. It also allows for a reconsideration of program goals and plans. When a formative evaluation reveals that a program has diverged from previous plans, those involved in the work can choose to revise plans to take advantage of new opportunities or return to previous plans in order to respond to current realities. Information from formative evaluation can provide input to future planning and implementation, thus forming the project's future.
The rapid feedback, documentation, and planning roles of formative evaluation make formative evaluation a useful component of reflective conflict resolution practice.
Types of Formative Evaluation
|
Mary Anderson provides six criteria for evaluating interventions. |
There are many evaluation tools -- observation, in-depth interviews, surveys, focus groups, analysis, reports, and dialogue with participants, each of which can be part of formative evaluation. Depending on the goals of the formative evaluation, it may emphasize one or more of these tools.
Within the range of formative evaluation approaches, there are four main goals for formative evaluation, each of which may be more or less emphasized depending on the program needs.[3] Each of these approaches to formative evaluation are briefly summarized below, with a focus on their applicability to formative evaluation of conflict resolution work in intractable conflicts.
- Planning evaluation. Planning evaluation clarifies and assesses a project's plans. Are the goals and timelines appropriate? Are the methods utilized to reach the goals appropriate? In addition, a planning evaluation can lay the groundwork for future formative and summative evaluations by developing indicators and benchmarks. In conflict resolution work, it is often useful to include a planning evaluation component in order to ensure that all stakeholders share common enough visions of the project plans. A planning evaluation can be a form of consensus building amongst those involved in conflict resolution.
- Implementation evaluation. An implementation evaluation focuses on the extent to which a program is proceeding according to plan. Information about ways in which a program is not proceeding according to plan can be used to either revise plans or to revise programming. In conflict resolution assessment, implementation evaluation can be a useful component to feed into a planning-focused evaluation. (Implementation evaluations can also be part of
Summative Evaluations
.) Where work is not proceeding according to plan, participants and facilitators can use an implementation evaluation with a planning focus to ask themselves why things are not going according to plan, and adjust plans or strategies accordingly.
- Monitoring evaluation. A monitoring evaluation is usually conducted by an outside evaluator during the course of a program. A funder may choose to monitor implementation of a conflict resolution project by visiting a workshop, checking in with participants, or talking with project personnel. For long-term conflict resolution work, a monitoring evaluation can provide a funder useful reassurance that money is being well spent.
- Progress evaluation. A progress evaluation assesses a program's progress. The project's unique goals should serve as a benchmark for measuring progress. Information from a progress evaluation can later be used in a summative evaluation. In conflict resolution work, a progress evaluation might assess attitude change part-way through a multi-year program, providing both feedback on what's working, and evidence of impact early on in a program.
Limitations of Formative Evaluation
|
Tamra D'Estree explains how evaluation can influence training goals and help maximize impact. |
Formative evaluation by itself does not meet the needs of most conflict resolution initiatives. As described in the evaluation overview module, there are roles for conflict assessment, action evaluation,
summative evaluation
, and research-based evaluation. Formative evaluation should be used as a complement to these other approaches.
Formative evaluation's focus is on improving work in progress. It should be employed to the extent that it improves work. Formative evaluation is not an end in itself.
Funding limitations often limit the intensity of formative evaluations. Where possible, formative evaluations should form part of funding proposals so that appropriate attention can be given to mid-course assessment and mid-course corrections in long-term conflict resolution work.
Finally, it should be acknowledged that, like all evaluations, formative evaluations may yield results that identify fundamental problems with a conflict management or resolution process. For example, when conflict management techniques are used to address a seemingly tractable dispute that is inherently part of an intractable conflict, a formative evaluation of the conflict management initiative might uncover the potential negative impacts of "successful" dispute management. In other words, where one initiative seems successful based on its own goals, formative evaluation could consider even the relevance of the goals themselves, and raise the long-term questions about dynamics which are perpetuating intractability by addressing some issues part-term.
How to Conduct Formative Evaluations
Formative evaluations can be conducted according to varied evaluation methodologies, but should usually incorporate the following considerations:
- Involve multiple stakeholders (see Action Evaluation)
- Involve multiple evaluation tools (see Overview Evaluation module)
- Consider plans, implementation, monitoring, and progress, as described above.
The particular shape of a formative evaluation will depend on the context, including the program being evaluated and the resources available. A well-developed form of formative evaluation is described in the Action-Evaluation essay. Other forms of formative evaluation may be more appropriate where it is not possible to involve all stakeholders as consistently as is preferable in a full action-evaluation approach. For example, a training team might discuss amongst themselves the progress in their week-long training every evening, but involve the participants in an explicit formative evaluation for a short session in the middle of the training. Formative evaluation does not need to take over a project and infuse itself into every aspect of the work, but should be incorporated to the extent that it can improve work.
Conclusion
Like all types of evaluation, implementing a formative evaluation in a conflict resolution program requires resources. However, these resources can be well-spent when a formative evaluation supports the reflective practice of conflict resolution, particularly in the long-term efforts that so often focus on seemingly intractable conflicts.
[1] See Scriven, Michael. "Beyond Formative and Summative Evaluation." In In M.W. McLaughlin and ED.C. Phillips, eds., Evaluation and Education: A Quarter Century. Chicago: University of Chicago Press, 1991.
[2] Quoted in Scriven, Michael. "Beyond Formative and Summative Evaluation." In M.W. McLaughlin and ED.C. Phillips, eds., Evaluation and Education: A Quarter Century. Chicago: University of Chicago Press, 1991: p. 169. Reported in Patton, Michael Quinn, Utilization-Focused Evaluation: The New Century Text. Edition 3. Thousand Oaks, CA: Sage, 1997: p. 69.
[3] These four approaches are discussed in more detail in Action Research on Web (http://www.uq.net.au/action-research/bd.html). The short descriptions which follow are based on information provided on that site.
Use the following to cite this article:
Nan, Susan Allen. "Formative Evaluation." Beyond Intractability. Eds. Guy Burgess and Heidi Burgess. Conflict Information Consortium, University of Colorado, Boulder. Posted: December 2003 <http://www.beyondintractability.org/essay/formative-evaluation>.
