Improving Social Inclusion Programmes Through Qualitative Research

SOCIAL INCLUSION PROGRAMMING IN SOUTH CAUCASUS

Despite recent economic growth, the South Caucasus region, and Georgia in particular, has consistently faced a wide range of socio-economic challenges. These challenges, which are underscored by prevalent poverty, include inadequate access to employment, entrepreneurial and educational opportunities and gaps in soft skills. To address this, diverse social inclusion programmes have been introduced across the region.

Action Against Hunger in the South Caucasus has been an important player in tackling economic inequalities and disengagement of vulnerable groups. Central to Action Against Hunger’s programming is the adaptation and implementation of the Employment and Entrepreneurship Shuttle methodology, developed by Action Against Hunger Spain. Through this methodology, social inclusion programming in the South Caucasus has yielded positive results, including increased participation and economic and social inclusion of vulnerable communities. However, given the complexity and multi-faceted nature of social inclusion programming, monitoring and evaluation of such programming has proven to be difficult.

MONITORING AND EVALUATION IN DEVELOPMENT PROGRAMMING

Various kinds of toolkits can be used to monitor and evaluate a programme, with Monitoring, Evaluation, Accountability and Learning (MEAL) teams often relying on quantitative data to inform, substantiate, and conceive programme findings. Most commonly, these toolkits are used to collect data on key socio-economic indicators at regular intervals before, during, and after programme implementation, to determine its success.

Generally, however, monitoring and evaluation is considered as being separate from programme activities, and programme implementers rarely go beyond quantitative data to collect other forms of data that may further inform programming. This implies, however, that several other components that relate to the success of a programme remain unobserved.

Increasingly, the complexity of development programming require the consideration of deeper and further reaching areas of impact that cannot be understood through quantitative research alone. Qualitative research, in response to this concern, helps to elicit deeper insights and explore participants’ behaviour, perceptions and understandings. However, the mainstreaming of qualitative approaches to monitoring and evaluation is often overlooked and remains a key challenge.

IMPLEMENTING A QUALITATIVE RESEARCH APPROACH

In 2019, Action Against Hunger in the South Caucasus decided to take on the challenge of going beyond quantitative data to implementing a qualitative research approach in its social inclusion programming.

The qualitative research approach, incorporating carefully-designed methodologies and tools, aims to ensure that the information obtained carries with it the validity and reliability that is necessary to inform effective programming. This involved following a number of key steps, ranging from clearly defining research objectives and the sampling methodology, to the adoption of appropriate data collection and analysis tools.
In implementing this approach, the South Caucasus MEAL team identified several key learnings that stand to inform efforts to collect and use qualitative research in future programming. At the heart of these lessons is the value of qualitative research in development programming.

WHAT HAVE WE LEARNT?

The South Caucasus experience elicited a number of useful lessons. These are described below and include ways to improve implementation of qualitative research approaches and steps that can be taken to ensure that qualitative data is used in an effective manner.

Adapt: A key takeaway is the importance of being open to adapting the approach throughout the research process. For instance, when the MEAL team recognised that programme participants had different linguistic backgrounds and competencies, adaptive steps were taken to ensure that all questionnaires, interviews, and discussions were translated to encourage meaningful participation.

Innovate: As with any approach to monitoring and evaluation, the experience of the South Caucasus reiterated the transformative potential of being innovative in the implementation of a robust qualitative research approach. For instance, the MEAL team learnt that pre-existing quantitative information on research participants could be uniquely leveraged to supplement qualitative research efforts; whether it is to stimulate discussions, formulate questions, or even determine a primary sampling pool.

Be Independent: Another critical lesson was the importance of guaranteeing the independence of the monitoring and evaluation process and ensuring that it remains distinct from programme implementation. A failure to distinguish between personnel and/or protocol in these parallel processes risked skewed findings—the programme team was less inclined to fully report on qualitative information from participants, and participants were less inclined to share their honest opinions with the programme team. In response, a distinct MEAL team was set up that was not directly involved in the implementation of programme activities. This helped to increase objectivity, eliminate bias and, in turn, improve the validity and reliability of research findings.

Support Implementation: In addition to improving the qualitative research process through flexibility, innovation, and independence, the MEAL team also learnt that qualitative data can be used to support effective programme implementation. In many cases, participants, or programme beneficiaries, associate with a development programme over several months and thus the programme becomes an integral part of their lives. Naturally, participants develop their own ideas and perceptions of how programmes could best function and what could be improved. The team learnt that such insight can be instructive and carry pertinent guidance on how a programme can be tailored and adapted to better meet the needs of beneficiaries.

In the South Caucasus, the qualitative information collected through key informant interviews (KIIs) and focus group discussions (FGDs), among other tools, confirmed that programme participants were eager to share their views and experiences. Qualitative information collected from a social inclusion programme, for instance, revealed that programme participants were dissatisfied with over-exposure to corporate volunteers as part of the shuttle sessions. The MEAL team communicated this feedback to programme implementers, encouraging them to make necessary adjustments.

Enrich Communication: The MEAL team also learnt that the adoption of a qualitative research approach stood to yield a wide range of unique findings that can enrich both internal and external communication efforts. For instance, qualitative information obtained from a social inclusion programme that aimed to improve employment skills tracked behavioural change to find that the programme resulted in participants reporting increased confidence in the use and application of skills to secure employment. Reliance on quantitative data alone would have only confirmed whether or not a participant was able to secure employment. Qualitative research can generate unique findings and deeper insight into the success of a programme which, in turn, can enrich learning and communication, be it internally among the programme team or externally in donor reporting.

LOOKING AHEAD

This article reflects the experiences of Action Against Hunger’s South Caucasus MEAL team in implementing a qualitative research approach in monitoring and evaluating social inclusion programmes. Read alongside a how-to guide on implementing qualitative research, the lessons shared in this review aim to provide a basis for the development of a comprehensive framework around mainstreaming qualitative research approaches in future programming.

In advocating the collection and use of qualitative data, the team is confident that this is a widely replicable approach that elicits valuable insights from programme participants and provides a unique participant-led perspective on what already works in a programme and what can yet be improved. Accountability to participants also obliges programme implementers to use qualitative findings to continuously refine programme design and implementation.

When qualitative research is valid and reliable, it can significantly improve programmes by providing a deeper understanding of participants’ perceptions and feelings. The South Caucasus’ experience has demonstrated that MEAL approaches to qualitative research can be strengthened through an openness to adapt, a willingness to innovate, and a commitment to independence. In doing so, the experience also confirms that qualitative research can be used to support effective programme implementation and enrich both internal and external communication. Action Against Hunger in the South Caucasus aims to continue to implement, refine and share new lessons from this approach as it consistently strives to better monitor and evaluate development programmes.

About this document