Over the last few weeks, the CLP’s Innovation, Monitoring, Learning and Communications (IMLC) Division has been busy preparing reports and analyses for a variety of purposes.  One of the areas that they have looked at is lessons learned from CLP, which is partly as a result of an external Operations Review / Lessons Learned study that DFID has commissioned.

 As a separate endeavour, I recently wrote an academic paper on CLP’s graduation approach and its evolution, which is due to be published shortly by the Institute of Development Studies (IDS – www.ids.ac.uk).

 These two streams of work collided the other day when CLP’s management team sat down to look at our recent graduation results (you can see a four-page brief here: http://bit.ly/Grad-2014-11). Between this and the work we were carrying out for the Operations Review, we spent some time reflecting on the various lessons that have come out of the graduation idea.

The evolution of graduation within CLP has taken quite a few twists and turns over the years.  It began during 2008 / 2009 under the first phase of CLP (which ended in 2010).

The CLP1 graduation approach mainly aimed at judging the extent to which CLP1 had achieved its objectives, rather than whether people had moved out of poverty.  From that perspective, it was little different from a logical framework reporting tool.

Things moved on substantially during CLP2 (2010 to 2016), with the team acknowledging that the CLP1 approach wasn’t illustrating participants’ journeys well enough.

The progress of the graduation approach from 2011 to 2014 involved at least three separate stages:

1. CLP developed a 2-stage process during 2011 and 2012. The first stage had three basic criteria, while Stage 2 consisted of a weighted scorecard of eight indicators (you can see the report here: http://clp-bangladesh.org/wp-content/uploads/2014/08/20120124-graduation-paper_formatted.pdf)
2. The 2012 Annual Review concluded that this was too complicated and recommended a simpler set of 10 equally-weighted indicators under six themes.  This was duly implemented from October 2012.
3. During late 2013 and early 2014, the graduation indicators and approach to collecting graduation data were again reviewed.  Some modifications to the indicators were made and the IMLC Division decided to collect both ‘snapshot’ graduation data within three months of each cohort completing their assistance, as well as continuing to track a panel sample of CLP participants who had already received support, to check on sustainability.  This approach was approved by the 2014 Annual Review and began from March 2014.

One learning from this quite lengthy process is that, if you ask any three development practitioners about graduation or poverty, you’ll get 20 different opinions!  It can also be difficult to know where to draw the line between review and implementation.  Three years to finalise an approach is fairly lengthy and could probably have been done quicker.  It is worth asking whether it needed to be formally reviewed by three separate annual review teams.

It also meant that, by the time the approach was finalised, we had a methodological problem.  How to retro-fit it to cohorts that had long finished CLP?

Our statistical wizards have succeeded, but we acknowledge that, for some indicators, we’ve had to ‘fill in the gaps’ with reasonable assumptions and analysis rather than directly-collected data.

One other strong learning that has recently emerged is over the purpose of graduation. Who and what is it actually for?

CLP’s position is quite clear: we believe that graduation is a useful management tool.  It helps us to aggregate statistics about key elements of our programme and our participants’ journeys out of poverty and work out who can be considered no longer part of the “extreme poor” group.

However, this raises a key question: what about the opinions of the people themselves?  CLP may well be satisfied with our approach.  But does it reflect the lived experiences of the extreme poor?

A very useful illustration occurred recently.  We had a brainstorming session to look at why only 17% of our participants, on average, were achieving the savings indicator (Tk 3,000 in savings to graduate).

 The answer turned out to be complex.  There were a whole range of reasons that could explain – partly or wholly – this finding.

Some of them were methodological; i.e. just the way that we collect the statistics.  For example, there may be an element of mis-reporting or mis-remembering going on – we don’t have bank statements to check against!

There are also timing issues. We know that there is a lean season, called ‘monga’ in Bangladesh, where the agricultural work dries up and families experience pressure on their incomes, expenditure, food security, nutrition and other areas.

Unfortunately, the time that we usually collect our statistics is bang-slap in the middle of this lean season.  It’s very possible that low savings may indicate that they’ve been used to keep people fed during monga!

But we also looked at the threshold of Tk 3,000.  Our reasons for choosing it were very valid from our perspective. These were mainly based around our experience of the funds required by a family to relocate and re-settle if they are eroded or forced to move by flooding.

What we hadn’t done is check with people, in detail, what their approach to saving was, the purposes that they valued savings for, and what thresholds they considered ‘good.’.

It seems that many people may prefer not to keep too much in the way of cash lying around.  Many will save until they have enough to invest in a new cow, a new goat or some more chickens. It’s possible that these ‘savings-like assets’ represent a more acceptable approach to saving for chars-dwellers.

So we came to the tentative conclusion that our Tk 3,000 threshold, while logical and sound from one perspective, maybe doesn’t represent a threshold or even a savings approach that entirely resonates with all our participants.

Although we’re not likely to change our indicator as a result, nevertheless, we’re intending to do some further studies and analysis to look into this tentative conclusion to see what we can learn from it.  With graduation, as with almost everything that CLP does, the learning journey never ends!

PS: the good news is that people still have very healthy savings once they’ve completed CLP, even if only 17% of households meet this indicator.

Our most recent control group shows that the average saving pot at the start of CLP is just Tk  129 – a few pence over £1. The average once they’ve finished is Tk 2,467 – nearly £21!

 A 20-fold increase in savings isn’t bad, even if we are aiming for a 23-fold increase…

19,633 total views, 1 views today