Monitoring Evaluation and Learning, often called MEL for short, is the practice of collecting and analysing data on a project or program. MEL enables project progress, impacts and performance to be measured and lessons to be generated for improvement of future work.
So many organisations with which we collaborate or provide support in Community Works struggle to achieve an effective approach to MEL. It is either something they feel ill-equipped to do, a reporting burden or a reluctant add-on to projects that are already underway. Many teams rarely appear that they are comfortably and competently managing MEL as a key part of their work.
For MEL to be truly integrated to project work, this situation needs to change. Part of the problem is that MEL is a professional discipline all of its own and one that is akin to research. But most program staff do not have a research background. When educators or health workers or community development practitioners take up a position with an organisation, they are often not acquainted with MEL because it has not been part of their formal training or even their previous experience. Faced with the complex task of defining indicators to measure, collecting data relevant to each and then analysing the data, it is not surprising that they appear overwhelmed. This is especially true where organisations do not already have their own data collection processes and systems.
So, what can be done? Here are suggestions that draw on our experience at Community Works:
Bring in community voices
Most staff working in community or non-government organisations, or for directly for government agencies, are accustomed to speaking regularly with the users of services or with participants in projects. Part of their job is to build trust and rapport with people in the community.
To make MEL easier, these strengths ought to be harnessed. In other words, some of the data collected should bring the perspectives of the community into the data by asking community members what their experiences have been since the project started. A small number of well-designed interviews and focus groups can provide high-quality data and rich insights into projects and programs.
Choose indicators with care
Some changes are complex to measure and may require extensive work to develop indicators and then work out how to collect meaningful data. Examples are community resilience or effective governance.
Rather than trying to design a framework for MEL that encompasses the full complexity of a subject, it may be valid to choose aspects that are more measureable and to focus on those. In the case of resilience, a measureable component is the level of confidence that people in the community feel about the steps they will take when a future emergency occurs. For governance, a key element could be the development of the skills of the chair and other members of a community board.
Use Theories of Change
A Theory of Change (or ToC) describes the key cause-effect relationships that underlie a development intervention. The discipline of focussing on the basic rationale for a project or program can help us decide with greater clarity exactly which data we need to collect.
Working with project teams, I have frequently concluded that more time devoted to discussing a ToC and less time spent on brainstorming indicators will lead to a better plan for MEL. Developing a ToC may be yet another discipline for an overstretched project worker to learn, but the benefits justify the effort. The discipline of developing a ToC can be valuable for anything from a single project activity, like a women’s group or a learning activity at a drop-in centre, to a major program scheduled over several years.
Less can be more
A few times recently I have been asked to review logical frameworks that represent the designs of projects or programs. Sometimes the column that includes indicators is very long. In other words, there are too many indicators. The impression is that the project team has thought of every single measure that could be applied but nobody has then culled the list to one that is manageable. One to three indicators per activity is usually enough for a logical framework. For a project or program as a whole, then between four and eight indicators ought to suffice.
We must always keep in mind that practising MEL costs time and money. Focussing on a smaller number of indicators is more efficient and less overwhelming for a team to manage than trying to apply a comprehensive set of measures that cover every aspect of the project. There is little to be gained from adding more indicators if the existing ones already enable effective MEL.
Direction of travel
For positive changes that can take years to realise, such as changes in attitudes and beliefs, it is unrealistic to expect a project of one, three or five years in duration to achieve those changes. In these situations, MEL should therefore concentrate on indicators that represent positive steps towards those longer-term goals. We call this the direction of travel. For example, for an individual or a group of people to show greater awareness of the health effects of smoking is an important step towards a change of attitude towards smoking.
Coupled with use of a Theory of Change, being clear about the direction of travel is valuable in reducing unrealistic ambitions of MEL, making it less daunting and more effective for even small organisations with limited resources to use effectively.
Rather than MEL being a burden for many teams and therefore done either poorly or not at all, it can be made easier. As a result, the value of doing it becomes apparent again. With data on the difference that projects and programs make, we are in a better position to know whether they are actually working. And we also learn why that might be the case and how to improve our work in the future.
