This article draws on the thinking in the CDRA’s most recent Annual Report – Measuring Development, Holding Infinity. In it, we argued that knowing the difference that we are making is itself a part of good development practice. In order to achieve this competence, certain processes and skills are required of organisations and individuals in development.  

In this article, I will talk about some of the issues raised in that Annual Report, and expand on those that relate to the ability of South African NGOs to know what difference they are making. The perspective shared is drawn from the CDRA’s collective reflections on practice, as well as my personal experience and reflection. I attempt here to offer a challenging picture of the state of South African NGOs and impact assessment. This is offered from the point of view of a practitioner who works largely with questions relating to the quality and efficacy of development practice, often as external evaluator.

The situation South African NGOs suffer from an inability to represent themselves. Simply put, we often know, intuitively, what difference our work has made, but we struggle to put that into words, to represent it to others. We lack the narrative when it comes to representing our work. This is not simply because of a difficulty with language, with writing, although we often feel tempted to claim this as the reason. It is also not simply because of the complicated reporting frameworks that we are often obliged to use, although, again, we sometimes would like to have ourselves and others believe that the problem lies here.

In fact, I believe that the real reason for our difficulty in describing the difference that we make lies a lot closer to home. There are at least two problems that we – as organisations and as individuals – bring to the table.

First, and despite South Africa’s relative abundance of professionals and wealth in formal education and training, as a whole, our development sector lacks an approach that is conscious, articulated and valued. By approach, I mean a rock-solid “take” on what it is we are doing that unites theory and practice, informed by a great deal of thinking, but constantly being put to the test in the field, and refined out of hands on experience.

This is not to say that individuals don’t practice well, developmentally, effectively. There are even individual organisations that pull off a lot of good work. But all of these little achievements remain isolated success-stories. As a whole, we lack a frame of reference, a practice out of which we work and to which we return with enriching learning and new ideas. This practice should go beyond simple textbook definitions and agreed upon conference resolutions; it should be a coherent and accessible reflection of a lived reality. However, instead of building this, we guard our successes jealously or trivialise them into simple techniques – as if good practice can, and should be replicated in blueprint fashion. When it comes to failures, we hide them very, very well.

One consequence of this lack of approach is that we have no frame of reference in terms of which we represent ourselves to our donors, to those to whom we must account. Each success must be proven, anew. Each failure explained in terms of its unique features. We cannot refer to successes or failures in terms of generally understood and accepted notions of what a developmental practice aims to achieve, because this doesn’t exist. In effect then, our ability to explain, describe and persuade others of our validity and impact is curtailed.  The second problem that we bring to the table is more straightforward, but equally limiting. It can be described as an inadequacy around planning and monitoring, a general refusal, or inability, to pay attention to the most simple of details in our work, and relate these to the bigger picture. To elaborate:

Individually and organisationally, we do not plan properly (and this despite endless, complicated training in the “technology” of planning, and obligatory annual planning workshops for most NGOs). The logic of a planned approach is extremely useful in development – but we don’t always think through the practical consequences of big visions and large timeframes and their relationship to day-to-day activities. The notion of planning as a living activity that happens in endlessly repeated cycles and at all levels of the organisation (right down to individual daily planning, not to mention “intending” each hour and minute) is generally absent. Because of this, we struggle to adapt plans, to work with fluidity, and to let go of them when they are no longer appropriate. More literally, strategic planning cycles seem to catch us unawares, they always have to be done in a hurry, they become trivialised, reduced to (more) bureaucratic requirements that have to be fulfilled, rather than seen as vital parts of the organisation’s thinking process.

Further, we do not keep good records (or hold our staff accountable when they do not supply them) and we do not use basic data as a tool for managing and accounting, we do not report sufficiently on the facts, the numbers. Basic record keeping is crucial. It tells us that something has happened – and this is a necessary precondition for knowing whether that “something” made any difference, or not. Without the big picture supplied by a planned approach, and without the details that track implementation, we are unable to supply credible accounts of the difference that we are making because we do not know, even if we can sense it intuitively.

The context

Our inability to say what difference we are making does arise out of and occur within a particular context. Amongst the features of this are the following: 

• South Africa’s unique political life and its continuing transition, now entering a second decade. At the level of practice, the approach that emerged out of the South African struggle at a grassroots level – to which progressive NGOs were generally party to – was extremely effective as a means of contributing towards achieving the end to Apartheid. Immediately after that, South Africa entered a period of negotiation and then legislation. Policy-making and education for democracy were the key activities of the times.
  
• It is only now that South African society as a whole is beginning to really face the challenge of implementation. Currently, those who practice within a legalistic framework are doing well, using South Africa’s progressive constitution and functional legal system to pursue implementation though the courts. However, social workers, fieldworkers, development facilitators and the organisations and institutions that employ these practitioners lag far behind in formulating a developmental approach aimed at delivering on the promises of South Africa’s transition. The methods of struggle are less and less appropriate to the demands of implementation, yet we continue to rely on them.
  
• A general lack of experience of  “partnership” for all parties concerned – NGOs and donors. The international move to seeing donor-recipient relationships as partnerships arises out of a past where the donor organisations tended to set the agendas of what happened in “local” situations. In light of this, the move towards partnership is a worthwhile attempt to shift the dynamic to one where there is more voice and agency coming from local partners. It might even be seen as a form of capacity building.
  
  In South Africa, however, the direction and dynamic is in precisely the opposite direction. “Local” agencies in other parts of the world are being asked to exercise more agency. In South Africa – considering our past relationship to donor funds which were freely given for anti-Apartheid activities – less working as free agents is being asked, more working as partners. Even 10 years on, this shift to greater collaboration and accountability remains a challenge for many South African NGOs.
  
  Equally for donors, relationships that provide fulfilling partnerships remain, on the whole, elusive. This is partly due to recipients’ struggles around accountability and responsibility, but is also due to donor’s own struggles with sharing. Who is the “we” when partnerships are discussed? Sometimes it does refer to the joint identity provided by a joint pursuit – the coming together of donor and recipient. However, and all too often, “we” refers to the donor, still, realistically, with the power of final decision, the right to withdrawal of money, the protection of “own” interests above the collective “ours”.
  
  Attempts at describing the difference “our” work makes often are foiled through a lack of clarity as to precisely who the “our” is. Seldom do we look at the relationship between giver and receiver and what has changed there. In seeking difference, we almost always look outside of ourselves, and seek it only in the “other”
   
• It is true that the logical framework – the dominant approach and tool for seeking and describing difference made - is unwieldy, complicated and over-bearing. More often than not it inhibits, rather than enhances our ability to describe the difference that we make.  Nevertheless, in the absence of our own approaches that might substitute for the “logframe”, objections to it appear rather thin.
   
• The project approach makes it hard to describe the difference made. While projects might be the most direct way of securing funds, the reality is that they tend to be a part of longer term and more comprehensive interventions. Describing the difference that projects make, very often demands that a greater “whole” is described. To do this, can feels laborious and rather futile, when all that is being asked for in the project report is “outputs” and “outcomes”. So we oblige with the dry accounts and leave the real stories out, thus obscuring the difference that a project can actually make, or even just contribute towards.
   
• We live in times of “outsourcing”. Increasingly, functions that were performed from inside of organisations are being sectioned off and outsourced as discrete “deliverables” to external contractors. Research is commissioned before strategic decisions are taken. External consultants are asked to facilitate ordinary strategic planning processes. A contractor maintains information technology systems. Logframe experts re-write proposals for organisations that can’t get their heads around the matrix. Outsiders are given short-term contracts to implement elements of a strategic plan. Still others are given the task of measuring its effect. Management of whole organisations can even be outsourced.
  
  Some of this outsourcing enhances capacity and impact and is an efficient way of getting things done. Other comes perilously close to outsourcing even the thinking of an organisation. When NGOs ask others to do their work they might well ask if they are not giving away that which is essential to their task. They might ask if they are not giving away the strategic initiative and capacity to really see a situation, make strategic choices about it and harness the resources needed to intervene towards achievement of a specific – political, value-laden – objective.

 The consequences

There are many consequences to the situations described above. In the realm of knowing and describing the difference that we make, one key consequence is an over reliance on the external evaluator. External evaluators themselves do not always have a solid practice. They may be well-positioned individuals, trusted by those commissioning the evaluation, or even people with a strong development practice themselves. However, none of these attributes necessarily equip an evaluator to find out and describe the difference that another organisation has made.

Despite this, the external evaluator has inordinate power in our lives. This is partly because their report will go to our donors and, attempts at forming partnerships aside, it is our donors who will use these reports as a form of external social audit, and it is upon their results that decisions about future funding will be partly based. But this is not the only reason for the external evaluator’s power. The real reason they hold so much sway is described at the start of this document. We are unable to describe what it is that we do and therefore, we are also unable to describe its effects. The external evaluator fills that vacuum in us. If they do not come up to scratch, if they do not craft words describing us in ways that confirm our intuitive beliefs about ourselves, we are left terribly upset. But also speechless and therefore, helpless to correct the situation, to change the impression created by others about ourselves.
 

The need

To conclude, I will not go into too much detail here as to what is needed of us as individuals and as organisations. A detailed discussion is offered in the Annual Report mentioned at the start of this article. But to emphasise two key points:

The first thing that is needed of all of us is to build proper relationships – between donor and recipient; between NGO and client community; amongst all of ourselves inside of our organisations. These relationships should not be especially ceremonial (images of the presumptuous donor, the fawning recipient, the powerful Director who never enters the field and the grandiose fieldworker who resorts to threats to control “their” community loom). Rather, robust relationships characterised by human warmth and mutual understanding (including good debate and disagreement) should be sought. We are all a part of the greater development project and to that extent we share common aims. It is the scope and detail of that commonality that must be thoroughly explored and mapped. Out of this, real impact can begin to be described and heard, and external evaluation put in its proper place.

Second and as a simultaneous pursuit for all of us, we need to build a proper development practice – including an ability to provide the narrative. We need to find the ways of representing ourselves - firstly, to our selves - then also to our donors and to other stakeholders. This holds true for donors, for recipients and also for those who are called “ultimate beneficiaries”. In the end, we seek development in all of ourselves, and this might well be measured in the extent to which we are able to engage in healthy relationships and make our voices, and stories, heard inside of those.

Here, the challenge of building a truly developmental field practice, distinct from instrumentalist approaches to organising, poses a special challenge for South African NGOs. If we are unable to rise to this challenge, then South African NGOs in particular are fated to remain somewhat marginal as they contribute less and less to the task of achieving implementation, dormant until those situations that require their special abilities to work as activists emerge again.

The Community Development Resource Association (CDRA) was  established in 1987 as a non-profit, non-governmental organisation (NGO) to build the capacity of organisations and individuals engaged in development and social transformation. We are based in Cape Town, South Africa and work mostly in Southern and East Africa.

Email: vernon@cdra.org.za Webpage: http://www.cdra.org.za
P.O. Box 221, Woodstock, 7915, South Africa
Telephone: -27 -21 462 3902
Fax: -27 -21 462 3918