‘We are not researchers’ – Making M&E accessible

‘We are not researchers’ – Making M&E accessible

by

‘We are not researchers’

Making M&E accessible

Steve Fisher

The increasing need for evidence of the impact of investments in social development, health and education programs places pressures on organisations and their staff. This is not unreasonable, but for many people, monitoring and evaluation remains an obscure subject, distant from the day-to-day activities of working with clients.

Community Works often runs introductory training workshops on monitoring and evaluation for health workers, NGO board members or groups of development professionals seeking to strengthen their work in this area. Sometimes participants make clear their difficulty with the subject. ‘We are not researchers’ they say. ‘Çollecting and analysing data is not something we have done before’. Or, they sometimes imply, is that the reason we became health or community workers in the first place.

We have learned from these experiences. As a result, we try to develop materials and deliver training sessions that follow four key principles:

Modify language

The phrase ‘impact assessment’ often throws people off course straight away. We have found that ‘measuring change’ is a good way to talk about monitoring and evaluation because it leads to a conversation about what are the specific changes a project or program intends to achieve and how can we best measure those changes. In social development, usually the answer is to ask people good questions. The same applies to other terms, such as data (equals information).

 

Desmystify the subject

Monitoring and evaluation is research. But that doesn’t mean the subject has to be clouded by research process jargon. For example, data analysis can be broken down into a process of grouping responses to interviews or surveys into key messages, trends and ideas. We often suggest that teams measuring change present all their data on the wall or on a big table, so everything can be seen together. That makes it easier to spot key insights and other information.

Focus on the process

When an electrician or a plumber comes to my house to repair or install something, the technical language they use is often a barrier to me understanding what they are going to do. We have addressed language above. But not knowing the process also leaves me clueless about their work. The same applies to monitoring and evaluation. In supporting better practice, one of the most useful ways we have found to support people is to set out the steps required. We sometimes present the process visually, as a flights of stairs for example. Once people can see the steps, the whole process seems more manageable.

Build on the strengths of participants

Often, qualitative information is best collected through, for example, interviews and focus groups. It sounds obvious, but the backgrounds of certain professional or community groups can be very good preparation for this kind of work. For example, health workers are often skilled in putting people at ease and asking questions in a supportive and encouraging way. This means they are often very effective, once given the chance to practice and guidance on the way to facilitate a focus group, for example.

In our training programs, we like to introduce practice early on. We find that participants who might have approached M&E with indifference or fear, become animated and interested when they see how effective they can be in collecting information through talking to people.

The forest and the trees in facilitation

The forest and the trees in facilitation

by

The term facilitation refers to methods and processes that enable people to collaborate on a given subject, especially where there might be complexity, different perspectives on a topic or even disputes and conflict.

I was recently invited to prepare a detailed tender submission for a role leading a facilitation team for a large and complex workshop. The client asked for responses to different scenarios and descriptions of how particular goals would be achieved through the facilitation. An occasional problem in facilitation is that the individual facilitator is given too much prominence in the process, as if they are some kind of motivational speaker. So I liked the way the tender concentrated on the proposed methods and the effectiveness of the process, a point that led me to write this article.

The work of convincing a client of the value of the particular approaches is a valuable exercise. It made me think in greater depth than usual about some challenges in facilitation, especially what works and why. I think of this as seeing both the forest and the trees. While an idea for facilitating a session might be sensible and proven, we need to have a clear sense of why we are choosing it. Some facilitators talk about ‘micro skills’, meaning the things you do to help a process along, such as the way the facilitator starts a session, how active listening is encouraged and ways to manage unevenness in participation. These are skills used to respond to what is actually happening at a particular moment in the process.

We need to combine these micro skills with a broader set of fundamentals for making the process effective. Here are some examples:

Getting the mood right

Groups participating in a process are often diverse and people may be in a setting to which they are unaccustomed. So they need to feel welcome and comfortable rather than sensing that they are being pushed through a program. The importance of setting the right mood and tone for the process, from the beginning, is often undervalued. The first session in each day, especially on the first day, should be lively and warm so people feel engaged at the start. Importantly, it should explain the purpose of the sessions that day and how the results are planned to be achieved. The tone should be one of relaxed productivity.

Straightforward language

A common mistake in facilitation is that the facilitator feels he or she has to ‘perform’, meaning that they interject too much, dominate proceedings or over-elaborate what they say. It is much better for a facilitator to be economical in their contributions, providing information using plan language. Explaining exercises visually and writing key words and phrases on a whiteboard is always a useful approach.

Inclusivity

Participants are different in their expectations of a workshop. Some will want to ask questions and make comments more than others. A key principle is to find a way for everyone to contribute in a way that suits them. So if people are nervous about speaking up in English, then some sessions will be more effective if they invite contributions on cards or visually on flipchart paper. Sometimes participants can feel like passive recipients of a plan developed in private by someone else. Instead, the thinking that underlies each day and its proposed methods should be explained by the facilitator. Bringing people along, and inviting their feedback, makes for a more open, inclusive and shared endeavour.

Judging the right level of active time

I have heard the complaint ‘I was just settling into thinking about the subject and then the facilitator had us running around like chooks’. This is fair comment and points to the need for careful judgement of the balance within a process between practical activities and listening time. People often want to be active, for the program to keep them moving around and doing group exercises. Most participants only want to listen to the same voice speaking to them for short periods before the next program component commences. But there still needs to be time for reflection. Getting the balance right is critical to ensuring participants are full engaged, encouraged and challenged.

In Community Works, we have recently developed some new methods and techniques for use in intensive workshops for Aboriginal health promotion workers and through teaching we have been doing on conflict resolution at James Cook University. I will share those methods and the learning that has come from them, in a future article.

Straightforward language

A common mistake in facilitation is that the facilitator feels he or she has to ‘perform’, meaning that they interject too much, dominate proceedings or over-elaborate what they say. It is much better for a facilitator to be economical in their contributions, providing information using plan language. Explaining exercises visually and writing key words and phrases on a whiteboard is always a useful approach.

Inclusivity

Participants are different in their expectations of a workshop. Some will want to ask questions and make comments more than others. A key principle is to find a way for everyone to contribute in a way that suits them. So if people are nervous about speaking up in English, then some sessions will be more effective if they invite contributions on cards or visually on flipchart paper. Sometimes participants can feel like passive recipients of a plan developed in private by someone else. Instead, the thinking that underlies each day and its proposed methods should be explained by the facilitator. Bringing people along, and inviting their feedback, makes for a more open, inclusive and shared endeavour.

Judging the right level of active time

I have heard the complaint ‘I was just settling into thinking about the subject and then the facilitator had us running around like chooks’. This is fair comment and points to the need for careful judgement of the balance within a process between practical activities and listening time. People often want to be active, for the program to keep them moving around and doing group exercises. Most participants only want to listen to the same voice speaking to them for short periods before the next program component commences. But there still needs to be time for reflection. Getting the balance right is critical to ensuring participants are full engaged, encouraged and challenged.

In Community Works, we have recently developed some new methods and techniques for use in intensive workshops for Aboriginal health promotion workers and through teaching we have been doing on conflict resolution at James Cook University. I will share those methods and the learning that has come from them, in a future article.

Overcoming isolation; why support groups are crucial for managing mental illness

Overcoming isolation; why support groups are crucial for managing mental illness

by

I used to volunteer with a group for men recovering from mental illness, which was organised by a local mental health care organisation.

I used to volunteer with a group for men recovering from mental illness, which was organised by a local mental health care organisation. We would meet every Wednesday at lunchtime and usually around six or seven men participated along with a facilitator. The facilitator would suggest some ways to make best use of the time, including games to help with building memory, topics for discussion and ways in which people could interact with each other in a positive way. The group was designed to be a safe and confidential place in which members could feel free to speak their minds and share personal information and experiences.

I noticed a couple of aspects of the group that taught me a lot about mental health and made me want to do more work in this field. The first was that all conversations tended to lead to one subject; work. We all defined ourselves through our occupations and felt pride in the kinds of work we were able to do and how many hours a week we were working. Progress was celebrated in a quiet way: for one man to express happiness that he was now doing three four-hour shifts a week inevitably left others feeling unhappy that they could not yet return to work. I quickly became used to the subject of work coming up during the first few exchanges and remaining a key topic throughout the meeting.

The second aspect of the group that woke me up to the realities of mental illness was the obvious isolation that everyone was experiencing. It wasn’t that they were without friends because most people associated with the organisation were visiting the drop-in centre most days and meeting others in a similar situation. What I noticed was a more deep-seated separation from the world outside mental health care, as if somehow the men in the group were cast adrift from the rest of the community because people didn’t really know how to talk to them anymore.

The value of the weekly conversations was undoubtable. They enabled the men to talk about issues they faced and helped recover skills and abilities they had lost or which were dormant, including having a sociable conversation with a group of other people. One man talked about improving his arithmetic skills and getting back to playing cricket again.

At the same time, I wondered what the rest of the week was like for participants. I went back to my office and a busy work environment, while they carried on with the process of recovery, often a solitary journey.

The group taught me about important factors that aid recovery and management of mental health issues. A greater number of self-help groups with experienced facilitators are a way of the mental health system embracing people recovering from illness and keeping in touch with them. We also need people in the wider community to be much more aware of mental health and to be willing to help those affected to overcome their isolation. Understanding the central role work has in our lives, and helping those who want to work, helps them regain a sense of self-worth.

The weekly sessions in which I participated were similar to the groups that BasicNeeds offer as part of its global model for recovery. BasicNeeds’ work in low and middle income countries involves encouraging participants to be a part of a self-help group, as it is through these groups that members get social and economic support.

I decided to join the self-help group in my area so as to join forces with people, lend a hand in farm production and raise awareness about mental health in my community. The group has changed the perception of people in the community regarding mental disorders and have shown the community that they are capable of improving lives…

– BasicNeeds self-help group participant.

The groups offer ongoing understanding and support for the conditions participants are living with, they build confidence to help them advocate for a better understanding of mental illness with their community and they help them to prepare for employment. Within these groups, people are listened to and are able to express themselves. It is a step towards individuals developing ways and means to manage their mental health as well as to advocate for their needs.

Effective governance; insights from recent training courses

Effective governance; insights from recent training courses

by

Starting meetings in the right way.

Ninti One has been conducting training in governance for voluntary committees in remote communities that are participating in the Stronger Communities for Children program, which is Australian Government supported. Working with colleagues at Ninti One, I have developed the materials and delivered the first few sessions in locations in the Northern Territory.
All along, we have wanted the training to be interactive, meaning that people get the chance to practice governance skills rather than just watching a presentation. Thanks to the willingness of committee members to have a go at the various exercises we have developed, some interesting insights and lessons on governance training have come out of the work. I share a few of them here.

Starting meetings in the right way.

It can be easy to think that board or committee meetings simply happen once everyone has shown up. The reality is that the start of meetings can be a messy and confusing business. Through the training, we noticed that an effective chairperson makes people welcome, sets the right tone for the meeting and gets the work started. Practising how best to start meeting makes all the difference.

Visuals replacing words.

Too much paper and too many words can sometimes get in the way of good decision-making. We found that summarising the main subjects for discussion in 1-5 words in a series of circles on the white board, with space for other people to draw arrows or make notes, provides a simple focus. In one case, the discussion really kicked off, with members of the group coming to the front of the room to explain their views on each subject.

Getting everyone involved.

Not everyone wants to speak in front of a group, which is fine as no committee needs to be full of public speakers. But still their views should be taken into account because everyone at the meeting is there as a decision-maker. We found that breaking up into small groups of 2-3 people to discuss particular topics really worked well. Here the chairperson has an important job in suggesting the right mix of people in the groups and making sure the report back from each one is clear and understood by everyone else.

Handling conflicts of interest.

A conflict of interest is not easy for anyone to define clearly, but most people know one when they see it. I describe it as a situation when a person’s role as a committee member is hampered by personal or family interests in a subject the committee is discussing. In small communities, conflicts of interest are inevitable. Through the training, we found that defining what it means, using examples and then agreeing a clear procedure where members declare their conflict of interest seem like a big step forward. Along with nepotism and confidentiality, the subject benefits from open discussion.

Improved and more effective governance is critical to remote communities. It has been a subject of much work by government, NGOs, researchers and communities themselves over many years. Ninti One is developing a sound approach to governance training that I hope will become more widely available in time.