All development projects and programs need and deserve to be evaluated. A good evaluation measures impact, generates new knowledge on program design and provides insights on how to improve the work. An even better one engages participants and beneficiaries in the research required to conduct the evaluation.
Sometimes an organisation will want to achieve some of these benefits but without a full evaluation. The reasons may be varied, including that their program is not fully implemented, they are trialling an idea with limited resources or they prefer an approach that is more exploratory than a typical robust third-party evaluation. In our experience, situations in which a more limited review or analysis of a project is being sought by an organisation are quite common. While a rigorous independent evaluation is valuable, there is also a role for more modest processes. The following examples come from our work:
Perspectives of users
The Eritrean Community in Australia asked Community Works to provide an overview of the views of the participants in their programs and the users of their services. We conducted surveys and interviews with a cross-section of people and provided the results in a summary report that enabled the organisation to gain insights they could not otherwise easily achieve.
Review of program strategy
Engaged by Prahran Mission, we undertook a short review of an existing program that was being replicated in a new location. The organisation wanted advice on what was working well and what could be improved. We reviewed program plans and reports, conducted interviews of key individuals and compared the strategies being used with good practice examples internationally.
Support to internal reviews
Organisations may consider they have the necessary skills to review a program, but require advice on the design and management of the process, the data to collect and how best to analyse it. We recently performed this kind of support role for an NGO that implements rural education programs, enabling them to research and review their programs using existing staff skills. The idea was not to replace an evaluation, but to reflect on the work to inform a new strategy being developed for the organisation.
Understanding a problem
Working for a client of Ninti One, we assisted the organisation to work out why a group of vulnerable people in remote Aboriginal communities were not making use of government services designed to assist them. By locating the individual residents and inviting them to talk to us about their lives, their needs and the suitability of the services on offer, we helped Ninti One produce the necessary analysis and recommendations that would enable improvements to the services to be made.
Case studies
It can be very useful for the story to be told of an individual or a family who has benefitted from a program. Working on a program review in Nepal, we went to visit former participants in a mental health initiative and asked if they could share their stories for the purposes of in the form of a case study. Achieving a story of this kind requires trust to be developed between the story-teller and the story-writer, which means that more than one visit is certainly necessary. The end result can shed light on ways in which an organisation can best anticipate future challenges faced by participants. In mental health, this may include overcoming discrimination from others that creates barriers to work.
In all these examples, and many others, a critical starting point is to define the question the organisation is seeking to answer. For the example above on understanding a problem, the question is obvious since the client needed to know why services were under-used. For others, bringing rigour to the process commences with identifying the right question. For example, a review might seek to know ‘how does our work compare with current leading practice elsewhere?’ or ‘how can we more closely meet the needs of the community?’.
As well as the methods described above, there can be a role for group work in mapping and analysing the work of an organisation. Please see our separate article on visual methods for working with community groups, which provides further examples.
