Critique of Rethinking Canadian Aid: Chapter 5 – Results, risk, rhetoric, and reality
I am doing a series of articles on the book “Rethinking Canadian Aid” (University of Ottawa Press, 2015), and now it’s time for “Chapter 5: Results, Risk, Rhetoric and Reality: The Need for Common Sense in Canada’s Development Assistance” by Ian Smillie. I like Smillie’s work, I like the way he writes and his take on things. I don’t always agree with the shortcuts he takes to explain things, but I do like reading his stuff.
Like others, Gilmore equates economic growth with development, and development with poverty reduction. That sequence is possible, and in some places perhaps even likely. However, the words and the concepts are not synonymous, and in too many poor countries the economy has grown on little more than the coattails of an extractive industry. Gross domestic product growth and averages often mask a reality in which real improvement to the lives of those at the bottom has simply not occurred, while in many cases the disparity between rich and poor has grown.
The BRICs that den Heyer referred to in Chapter 4 (Critique of Rethinking Canadian Aid – Chapter 4 – Power and policy) are the perfect example of this — overall per capita growth is high, but the resources are not distributed to all citizens even in the most ambitious view of trickle-down theory.
The answer to development is not one thing; it is many. The question is not whether the market is an alternative to aid, but whether aid can deliver on its part of the promise. The answer to that question seems not to be all that clear.
I’m curious if any of the other chapters will get into the numbers for measuring flows. I hope so, as there are lots of indications/place markers that would tie into a sophisticated numerical analysis.
So here is a second problem: Despite billions of dollars spent on development assistance, we seem not to have made much progress in reducing poverty. Where there has been progress — in places like China and other East Asian economies — it can be ascribed much less to aid and more to the phenomenon so beloved of aid critics like Gilmore and Moyo: the market. It is perhaps no wonder, then, that politicians attempting to make sense of their country’s aid budgets are demanding better and more evident results.
This is one of the short cuts that Smillie takes that frequently rubs me the wrong way. I don’t mean he’s wrong, but it does strike me as somewhat misleading. For example, there is an easy way to reconcile aid and market techniques — aid is about stopping social and economic backsliding, markets are about moving the economy forward (or at least growing it by attracting additional outside resources). Of course, there’s no proof — a huge problem in development writ large — but there are almost no countries where markets showed progress without a strong aid underpinning. I think the reality is somewhere in between — aid makes small or large differences in certain circumstances and not others; markets grow in some circumstances and not others, often having more to do with external context rather than internal strategies. If the economy is booming, every initiative works; if the economy is tanking, no initiative works. What’s more telling, and hard to measure is would there have been less benefit in booms or more disaster in busts without aid or the market initiatives?
If donors are serious about poverty reduction, one might think that a preponderance of aid would go to the poorest countries. Not so. In fact, only one-third of all ODA goes to the least developed countries and, if one takes Afghanistan out of that mix, the percentage drops to about one-quarter (OECD 2012).
Another short cut, and it is a popular one that the press and the Canadian public could get behind pretty easily. Popular, digestible, seems like common sense, and almost completely wrong. First, former PM Chretien had a very strong opinion about aid concentration and that opinion was not only strongly held, but also hard to refute as a strong aid premise — he would not sit around the table at a Commonwealth or La Francophonie and say “we will help you, but not you, you but not you”. We would help everyone. Not equally necessarily, but we would help all our “partners”. Which makes it very hard to “concentrate” on just the poorest. Second, what most of the public wouldn’t understand is absorptive capacity. People like to think you can drop any amount of money into a poor country and thus spend millions and millions, concentrating the most money. But those countries can’t do everything at once, they don’t have the local capacity, and unless you’re going to send development workers in like an invading army to “do it for them” (which would violate just about every principle of development), there are limits. When it’s a humanitarian crisis, sure, you can drop billions, but that’s not development. Third, development doesn’t stop at a border, although it frequently seems like it. Sometimes helping the poorest country might require helping the neighbouring country build its economy too so the region prospers. If you’re looking for a Canadian example, look at transfer payments between provinces — not all to the “poorest” province, but designed to raise the tides of all provinces. Are regional efforts effective? That’s a separate story, but it isn’t quite as simple a short cut to country concentration.
So before we get to the question of results in terms of the stated aim for development assistance — poverty reduction — we need to think about how much money is actually available for the effort. To do this, we should heavily discount or even remove:
ODA spent in pursuit of political, strategic, and security interests;
ODA spent in pursuit of commercial interests;
ODA spent on technical assistance;
ODA tied (directly or indirectly) to Canadian goods and services;
ODA spent on humanitarian assistance;
ODA spent on refugees during their first year in Canada;
the cost of administering the aid program, probably understated at $244 million in 2010–11 (CIDA 2013b); and
the cost of foreign student subsidies, estimated at $168 million in 2010–11 (CIDA 2013b).
It’s a great list. Clean, common sense for the common man. Except, well, I have to nitpick a little. Let’s look at “security” interests. Presumably, there is no argument that development cannot happen in a country that is being ripped apart by war. And there is also equally no argument that conflict has a nasty habit of not respecting borders. So, if Australia sees trouble brewing in those pesky island neighbours, presumably they would be right to consider conflict and war a destabilizing influence on the region that would / could / might / should affect their own country. In other words, they would have a strong national security interest in helping reduce such conflict. It might even induce them to pack off hundreds of aid workers, diplomats, and soldiers to the region to restore peace and contain the potential for sparks of war. Yet, does the motive dictate the classification as aid or non-aid or the result? Even in the individual psyche, we rarely do anything for a single reason, so is it right to say, “Well, your primary reason for doing x is your own interest, and therefore even if it leads to developmental results, it isn’t aid?” And if we treat it as a crime against development to have “other motives” (either primary or secondary), we do indeed have the mens rea of intent, but the actus reus may look a lot like development in another country. Aid, at its core, has at least two functions — to help the developing country and to help the global community of which it is a part. Even if all aid does is give everyone another trading partner in the end, there’s still a “commercial benefit” in the end.
Commercial interests are equally grey. Take radar satellite technology. I think, personally, that they represent one of the sectors in Canada that looks the most self-serving, supply-driven approaches on the planet. They come forward and say “Hey, we have this great technology for finding water, mapping geo trends, helping with urban / rural planning, environmental planning, etc.”. And they have projects ready to go, just write them a cheque. Sounds incredibly clear, it can’t be aid. Except it is no different than OxFam Canada coming forward and saying “Hey, we have this great idea for a project in Africa, and if you just write us a cheque, we can be up and running in no time.” One’s an NGO, so no “profit” motive, but is the “profit” the problem? Could we pay the profit portion out of another budget and pay the rest from aid? If we get a kick-ass, state of the art, environmental plan that knocks everything else out of the water, is the result that defines it as aid or the fact that the company developed the technology for other purposes? Because there are a lot of people eating food and using medicines that were initially produced for profit. Put more fairly, and maybe this ties (no pun intended) better with the tied aid, maybe it shouldn’t be a 0% or 100% choice, but rather a discounted amount.
I won’t spend any time debating technical assistance. A whole book could be written on not only what “is” TA but what the benefits look like, etc. Far too much to include here. I’ll settle for saying I would need a lot more argument to discount it 100%.
Tied aid is pretty complicated, and while we have lots of great statistics, I think most of them are meaningless. Take for example two trade-related projects (which by the way get classed as technical assistance, compared with identical projects in the social realm that get classed as simple capacity building), one that had a Canadian project manager so looked like 100% tied aid even though 90% was spent in-country on non-Canadians, and one that was run by a local organization in-country but used all Canadians for the expertise, so was 0% tied aid but 90% of the funds came back to Canada. The statistics and the systems are just not sophisticated to account for that kind of issue. Separately though, there are lots of good examples where the country wanted “Canadian expertise, Canadian goods” etc. — so Canada provides what it is asked for (knowing in some cases that the developing country is asking for Canadian so as to increase the likelihood we’ll fund it), but gets “dinged” because it was Canadian content. I don’t disagree that there’s an issue, but 0%/100% likely isn’t the solution either.
I have no real issue with discounting humanitarian assistance and refugee costs. There are some out there that make lovely convincing arguments for development content within both, but it’s a step too far for me.
Administration costs is a strange item, and defies the “common sense” argument. Let’s look at five examples of delivery mechanisms. Let’s say there’s a project for $10M in Vietnam, with admin costs of about $1M to manage and implement. For sake of argument, I’m going to hold it to 10% for all scenarios just so the numbers stay the same. Option 1 has CIDA design and deliver the project themselves. $1M for CIDA costs and the argument would be that’s not for the benefit of Vietnam so don’t count it. Option 2 would have a Canadian NGO do it with $50K in costs for CIDA and $950 for the NGO. Again, not spent in Vietnam, don’t count the $1M. Option 3 has a multilateral deliver it with HQ staff — say $10K for CIDA, $990K for UN org or international NGO. Don’t count the $990K or count it? Arguments either way (since it is now merged with the programming money). Option 4 has local NGO do it, with $50K for CIDA and $950K for NGO — local cost, local spent, so count $950K and leave out $50K. Or Option 5 with the government where $10K is in CIDA’s control, and the other $990K went to the Vietnamese government, but with no clear tracking since it was pooled funds.
Five options, same project, same costs, but the governance model for funding has five separate calculations for what the project cost. And the simple “common sense” approach would be relatively straightforward — you charge costs of delivering a project to the project, and developing projects are charged to the aid budget. Should it be 100%? Perhaps not. But I find little resonance with saying “we’ll count these costs here, but not these costs there”.
For the cost of foreign student subsidies, I would love to treat it the same as the refugee costs — it doesn’t “look like” development, therefore it shouldn’t be counted. Except CIDA used to do scholarships back in the day, with many of them in the Caribbean. We gave these Canada Scholarships to “deserving” students in developing countries, brought them to Canada, gave them a university education, and then they went home. A very strong “individual benefit”, not development. Except one of them became Prime Minister, and several others became Cabinet Ministers. It looks a lot like a long-term governance project that paid off huge dividends for relatively small investments. And if they don’t go home, but stay, there’s always remittances flowing back too. Sooooo, I can count a micro-credit or small-business grants if they set up the business in-country, but if we give them educational subsidies, and they go back or stay here and start businesses that lead to additional resources in-country, we can’t count it? Again, the short-cut looks great, and the line has to be drawn somewhere, but I’m not convinced 0% is the right line.
Despite the herculean effort, CIDA still had a hard time explaining its results. The government’s report, Development for Results 2010–11, is almost exclusively about what CIDA did during the year: activities rather than results (CIDA 2011).
I need to digress for a moment here. I’ve done a lot of different jobs in government. Policy development, horizontal coordination, multilateral relations, multilateral programming, bilateral programming, some information work, regulations, finance, etc. And while I was better in some areas than others, and some I was quite good at, there’s a different area that I have shown somewhat unique strengths. Corporate planning. Performance measurement. Evaluation, risk management, reporting. I say that my strengths are a bit unique in the domain because I don’t act like the typical planner. Most corporate types embrace their inner planner and want to go to the nth degree of detail. All the bits and pieces that Smillie talks about — logic models, performance measurement strategies, data collection templates, performance indicators, evaluation plans, etc. And when it comes to indicators, most planners want more, more, more. I don’t. I usually am asking for less detail, shorter explanations, simpler logic chains. “We did x in order to get y and we got z results.”
Reading Smillie’s analysis of the various bureaucratic templates and tools, I’m struck by three things. First, a very inelegant “no sh**, Sherlock” reaction. Yep, everyone can tell you that the reporting burden is too high, including everyone that works there and everyone who has worked with them. That’s not news.
Second, based on a bunch of work I did when I was at CIDA, including the 2002 OECD DAC review of Canada’s aid program, I can tell you that not only is Canada really bad at explaining the x, y, z above (they even struggle with x and y at times, and never z), just about EVERYONE in development has similar problems. The ones who don’t are either heavily focused on humanitarian work (heck, even Rick Mercer can calculate the benefits of a bed net), taking short-cuts (such as using percentages to calculate the results of multilateral funding), or heavily focused on short-term outputs (one step past activities, but still shy of outcomes).
Third, now that I’m mostly out of the development world in my day job, I’ve come to realize that most government departments generally suck at results reporting. Activities can be explained, sure. Logic chain to the expected outcome? Maybe. Actual results at the outcome level? Extremely difficult on an annual basis.
Let’s look at a simple development project, maybe one that was giving out small micro-credit-style loans. Let’s assume one per month, normal calendar year. Great, so at the end of the year, they did 12 loans. That’s the activity. Of course, they know why they did it, it was to improve recipients’ standard of living, and let’s assume that it’s measured simply by income. So we have “12 loans to raise income” as the first two parts. But with what results? Suppose it takes 9 months to pay off the loan and be sustainable (wishful thinking). But, the person who got the loan on Jan 1 was successful by October; February person by November; and March person by December. Therefore 3 successes. But the April person is almost successful, as is the May person. June can see the light at the end of the tunnel, July is optimistic, etc. It may be that all 12 will be successful, but if you calculate results for that year in January of Year 2, the only successes you have are the three ones who got their loan first. Or, in other words, a project with 25% success in the first year. In year 2, that project would look amazing — 9 successes from the previous year and 3 from current would give 12 successes in year 2 — 100% success! Except perhaps the second half of the year was difficult and those who received loans after July will eventually default. The real success in year 2 was maybe only 50% — and the other 9 results were from previous funding, NOT the funding from that year.
So, this leaves the aid agency with a dilemma — what results do you report? Annually makes almost no sense, and lots of people will say, “no problem, we’ll capture it during an evaluation in five years that will negate the annual lag issue”. Except almost everyone active in development tends to think development projects often don’t show results until year 5 or 6 anyway — or longer if they are education projects. Short-term outputs, but long-term outcomes. The impact that Smillie started with, and the focus on outputs with which he ended.
But all those templates didn’t come from nowhere. They came from hundreds and hundreds, if not thousands and thousands, of Canadian aid workers (inside and outside government) asking the same question — how the hell do we measure this? What tool can we use that will help us be able to report? We believe we’re making a difference, but how do we show it? So they invent a new tool. And another template. And systems to track metrics as well as the money.
It is the same motivation that drove donors to agree to the Millennium Development Goals — if we couldn’t measure progress and results on an individual basis, perhaps we could measure global collective performance (the ultimate outcomes!) and use logic chains to show how our projects contributed to that performance.
Again, I find myself liking Smillie’s prose, even agreeing with some of the general conclusions. But then a too-simple-by-half summary turns me off. I see the health of individual trees while Smillie is noting the forest is dying. It’s always a fun read, but I still want to move some of his lines around.
![](https://www.thepolyblog.ca/wp-content/uploads/2021/04/signature-01-happy-reading-e1618075576972.png)