Fresh Insights into Impact & Learning from Failure

Let me take you back to three decades ago. In 1984, David Pyle wrote his work “Life After Project" (A Multi-Dimensional Analysis of Implementing Social Development Programs at the Community Level, Boston, John Snow, Inc. 1984.)

His words still echo in development cooperation. This is especially true when we talk about impacts and learning from failures.     

“How many times during the last three decades of intensive development efforts has a demonstration or pilot project provided ‘the answers’ to a development problem? Everyone is flushed with enthusiasm and optimism. The model that proved so successful on a small-scale is expanded with the hopes of benefiting a larger portion of the population. All too often, however, impact decreases or disappears completely [once the project phases out].”

David is right. There are several examples that lack honest inquiry. We often read about successes, proudly packaged in glossy formats and heavily disseminated and presented. To be fair, this isn’t the whole story; there are also good cases of success. Others that failed are developing the culture of learning to improve their work.

We believe in and we also have examples on how development cooperation has brought about impacts to the lives of people – be it in the education, infrastructure, governance, health or other areas. In contrast to a decade ago, fewer people are now living in extreme poverty around the world. For the past few years, world’s unemployment rate also significantly dropped to the lowest level since the global economic crisis in 2008.

Yet, we also realize that challenges remain. This is especially true in making development cooperation more ‘human-centered’ – to squarely put people and their lives at the center of any efforts. Inequalities, for example, have expanded within countries; much work is needed to address the deficits of decent work in a rapidly changing labor market. Above all, sustainable and inclusive development isn’t just about achieving economic growth or addressing economic inequality (in absolute or relative terms). It’s also about improving human capital and increasing the voice (agency) of poor and disadvantaged women and men.

So, what to do about it?

And now comes the best part. As development practitioners, we play different roles and have the responsibilities to critically reflect on our practices, successes and failures. In the East and Southeast Europe Unit of Helvetas, we’re striving to contribute more to evidence generation through knowledge management and learning – both by critically reflecting on our work (and therefore improving) and sharing with others about what works and what doesn’t.

Here are few examples

First, together with Mesopartner and Voices that Count, we’ve recently started a regional impact assessment of three youth employment projects of the Swiss Agency for Development and Cooperation (SDC): RisiAlbania, MarketMakers Bosnia & Herzegovina and EYE Kosovo. Even though limited in size, the impact assessment surely contributes to critical reflection on key questions of impact and learning from failures, and supports significant changes that projects make in labor market systems.   

Second, we serve as ‘knowledge brokers’ through generating knowledge products by, for example, bringing together different stakeholders where we jointly blog on inclusive systems. Here’s what we’ve lined up for you in this edition of Mosaic:

• Check out our conversations with Katrin Ochsenbein from SDC Kosovo and Susanne Thiard-Laforet of the Austrian Development Agency (ADA) about their understanding of impacts, and the progress and challenge in development cooperation.
• Two academic institutions – the International Program for Development Evaluation Training (IPDET) of University of Bern and Centre for Development and Cooperation (NADEL) of ETH Zurich – tell us why learning and adaptation are important.
• Three partners from East and Southeast Europe – Impact Ventures of North Macedonia, Partners Albania for Change and Development and the National Alliance for Local Economic Development (NALED) from Serbia – have shared their experiences of development gains in the region and what needs to be done to achieve longer, bigger and better development impacts.

Third, we try to tie up things by developing and using knowledge management and learning strategy. Our aim is to support the systematic generation, documentation and sharing of knowledge and learning. We use several tools and processes: peer exchanges, case studies and thematic working groups. As Katrin Ochsenbein from SDC Kosovo has alluded to in her contribution to Helvetas Mosaic, the working groups on a range of topics in the region are a good initiative, because they allow discussions among peers, at the same level.

Understanding impact

Phew! This is a critical issue, and the hardest part. Understanding the type, quantity and quality of impact helps us figure out what works, what doesn’t and why. Of course, impact in development cooperation remains complex and nuanced. Our experiences show that it’s a multi-dimensional concept – it can focus on specific and concrete achievements and can also be about broader, positive changes.

Matthias Herr collaborated with Stefanie Krapp (IPDET) and Kimon Schneider (NADEL) and wrote on “learning and adaptation”. The three believe that “our definition of ‘impact’ needs to include the notion of causality that understands people and institutions as being part of a wider and complex system”.

What does this mean for development projects?

Clearly, there’s a need to (re)think contributions to impacts, not just as interventionist that treat complex problems as simple input-output models, but as complementary and facilitative. At its core, the link between a development initiative and its impacts (i.e. the causality) is about making a constructive and a significant difference in the lives of people. As Violeta Jovanović, Executive Director of NALED in Serbia puts it, impact is about real change “in our members and partners in terms of cutting red tape and creating better legislation to improve competitiveness and the standard of living in Serbia.”

One important measurement of such a contribution is systemic change. But what’re we talking about when we say, “systemic change”?

With the risk of oversimplification, it’s about a wider and a larger impact both in scope and quality. It goes beyond few farmers, young people, enterprises and municipalities. The impact can be material, behavioral or in the practices of individuals, businesses, public sector agencies, civil society organizations within a system.

What’s equally important is how development projects use different sources and types of evidence to tell a coherent story of impact. In her contribution to Mosaic, Susanne Thiard-Laforet from ADA neatly summarizes this point: “It isn’t only about defining impact, it’s also about telling the impact story in a way that responds to the needs and requirements of various stakeholders.” 

Walking the talk to achieve better impact  

We’re changing and improving the way we do development in order to be more effective. No matter which system a development project is addressing – be it financial services, decentralization processes or sanitation technologies – there needs to be a vision on how this system can work in the future for the benefit of people and, particularly, poor and disadvantaged groups. Putting the vision in practice also requires understanding better why an education, a health or an advocacy system underperforms or fails.  

Facilitation pays off. The evidence is growing, as development practitioners learn and improve their work in complex contexts. Linear, largely pre-planned initiatives are poorly suited to complex problems and contexts, and therefore don't necessarily bring about long-term impacts. For this, development projects use adaptive management for testing partnerships, innovations and piloting activities through navigating complexity. Simply put, this means the ability to change strategies, plans and activities quickly in response to new information or signal to become more effective.

The experience of Partners Albania confirms this reality: “Sometimes, you may be more driven by the numbers that may or may not be sustainable in the long run. You might produce a certain number of employed people and even achieve changes in the legal framework – but, we don’t know if the jobs will be still be there and if the legal framework will be implemented after the completion of the projects.” 

Taking development seriously as a multi-stakeholder initiative

Achieving impact lies outside the sphere of control of any single development project or organization. Therefore, development cooperation is a multi-stakeholder initiative. Long-term and larger-scale impacts happen when partners – be it a private sector enterprise, a civil society organization or a government agency – take up development initiatives and leverage them for better impacts. For donors like SDC in Kosovo, this means all the stakeholders are convinced of the benefits and that there are clear and shared visions of objectives.

As strategic partners, it isn’t just contextual knowledge that locals in countries where development projects are implemented are capable of and ready to contribute. The knowledge in these countries constitutes understanding the complexity of development challenges and proposing and carrying out durable solutions. The story from North Macedonia by Igor Mishevski and Elena Ivanova, the founders of Impact Ventures, tells this accurately: “you end up telling your partners about your solution, but this doesn’t mean that you listen to them! And not listening always means not understanding, leading to incorrect solutions.”

Nurturing the learning culture

Oops moments? Yes. All the above points make no sense if there is a lack of knowledge and learning culture. “Of course, failure is normal,” stresses Katrin Ochsenbein from SDC Kosovo. Susanne Thiard-Laforet from ADA agrees: “It’s human to make mistakes or having to face challenges – regardless of whether they’re self-caused or external.”

Yet, in most cases, development initiatives suffer from the deficit of ensuring that learning is durable. Two reasons are worth mentioning.

First, learning is taken as an add-on and not part of our work. Therefore, the culture of learning is weak. This leads to such a culture eating any knowledge management and learning strategy for breakfast! Fear, embarrassment, and intolerance of failure drives our learning underground and hinders innovation.

Juliana Hoxha, Director at Partners Albania, believes that the learning culture needs to happen both at the individual and institutional levels. Critical reflection and learning from failures need “an honest and deep personal reflection on your own actions. For me, at the organizational level, we try to do this by way of regular assessment of our progress that is the first step to minimize failure. How do you work? Do you create an added value which may or may not have been intentional or something negative?”

Second, even if learning happens, it’s just owned by development projects. There are scant cases of development projects working with actors such as universities. Our exchanges with educational institutions (e.g. University of Bern and ETH Zurich) confirm that how knowledge and learning is used matters the most. For this reason, learning and adaptation are necessary and accompanying actions that turn knowledge into something useful.  

Additional sources

• The ‘Oops Moments’: Why Should We Care About Discussing ‘Failures’?
• Three Considerations when Measuring “Success” in Development Cooperation: A Conversation with Marcus Jenal
• How Chasing Fixed and Large Outreach Targets Hinders Scalable and Sustainable Impacts
• “Systemic Change? What Are You Talking About?”

This article appeared in the December 2019 issue of  Helvetas Mosaic. 

Subscribe to Helvetas Mosaic

Helvetas Mosaic is a quarterly published by Helvetas Eastern European team for our email subscribers and website visitors. Our articles explore new trends and fresh ideas of international development work in Southeast Europe.