A $3.8 trillion mistake, and why it (mostly) doesn’t matter
The controversy over mistakes in a key paper used to justify austerity reminded me of what seems to be another simple mathematical mistake in a high profile paper. The authors of this paper tried to tot up the total value of the world’s ecosystem services—the ways that humans benefit from the natural world. This kind of valuation is pretty controversial: critics feel that putting a dollar value on nature misses the point and leaves some ecosystems vulnerable, while advocates argue that it’s the only realistic way to include nature in a discussion that all too often just looks at ‘the economy, stupid’.
Costanza et al. ran the figures for a whole range of ecosystem services, and estimated that we get some $33 trillion of value per year. The paper was published in Nature, and according to Google Scholar, has since been cited some 9488 times. I won’t discuss how they got all the numbers and what they mean; if you’re interested, have a look at the paper itself, and critiques such as this and this. My focus today is more specific.
Looking at the breakdown of values for different services and biomes (table 2), by far the largest single value is for nutrient cycling, described in table 1 as “storage, internal cycling, processing and acquisition of nutrients”, for example “nitrogen fixation, N, P and other elemental or nutrient cycles.” That’s mostly from marine systems: the total value of nutrient cycling by marine ecosystems appears to be $15.3 trillion per year, a little under half of the grand total. To get behind that number, we need to dive into the supplementary information. The relevant section reads:
We assumed that the oceans and coastal waters are serving as sinks to all the world’s water that flows from rivers, and that the receiving marine waters provide a nutrient cycling service. If we assume that roughly one-third of this service is provided by estuaries (Nixon et al. 1996 in press) and the remainder by coastal and open ocean, (assume 1/3 by shelf and 1/3 by ocean), then the total quantity of water treated is 40 x 1012 m3 y-1. Replacement costs to remove N and P were estimated at $0.15 – 0.42 m-3 (Richard et al. 1991 as quoted in Postel and Carpenter 1997). Thus, the replacement cost for each biome’s (1/3) contribution to the total value is $2.0 x 1012–$5.6 x 1012. By hectare, the value for ocean (32200 x 106 ha) is then $62.1 – 174 ha-1 y-1.
First, where does that total quantity of water come from? It looks like it should be the total flow of the world’s rivers, and this Russian paper from 1993 confirms that, putting total river runoff at 42,700 km3 per year, about the same as the volume given. I’m not entirely sure about this way of calculating replacement value—if we had to recycle those nutrients ourselves, wouldn’t we find a more efficient way than filtering them out of all the world’s rivers?—but let’s accept the assumptions for now.
Multiplying the total volume by the estimated treatment values (15 to 42 ¢ per cubic metre) gives us $6.0–16.8 trillion, a range with a midpoint of $11.4 trillion. That’s quite a bit lower than the value $15.3 trillion total we got for nutrient cycling in the oceans above. Where has the extra come from?
As the quotation mentions, the authors assume that three biomes—estuaries, the continental shelf and river estuaries—each perform a roughly equal amount of that nutrient cycling. So the total value is split into three chunks of $3.8 trillion, one assigned to each ecosystem. But then a fourth chunk of $3.8 trillion turns up, assigned to seagrass & algae beds (and by itself making those the third most valuable biome per hectare). The notes for nutrient cycling in this biome just say “For calculation methods, see notes for Ocean.” I can only guess that the authors decided that seagrass and algae beds had an important role in recycling nitrogen and phosphorus, but forgot to recalculate it to split the total over four biomes.
I’ve checked over this several times, and when I first spotted it, I bounced it off one of my professors, although unfortunately I no longer have the e-mails. If I’m wrong, I hope the authors will accept my apologies. But even in that case, I don’t think that the information in the paper and the supplementary information clearly justify these numbers, which make up the biggest part of the headline figure.
Of course, unlike in the austerity paper, this doesn’t really affect the conclusions of the study. Whether the central figure is $33 trillion or $29 trillion, it’s clearly enormous: the authors highlight that it’s more than the world’s GNP ($18 trillion in 1997, when the paper came out). And that number is just the centre of a range of $16–54 trillion, so there was already plenty of uncertainty. More importantly, it’s probably an underestimate, because of all kinds of services and factors that couldn’t be included in the analysis. It’s clear that we benefit immensely from the natural world.
So is this $3.8 trillion dollar slip wholly unimportant? Why am I writing about it? Well, I don’t think it’s a great testament to the peer review process. All the information to spot this was there in the supplementary information, but even for a paper in the world’s most prominent journal, reviewers apparently didn’t reach for a calculator and check through the numbers. Happily there are moves afoot to improve reproducibility, and people are working on better tools for scientific computing. But how many other papers out there have simple mistakes in the numbers?
But I’d prefer to look at this more optimistically. Like Thomas Herndon, who dug out the Excel mistake in the now-infamous Reinhart & Rogoff paper, I spotted this while I was an undergraduate, reading about ecosystem services for my conservation module. Checking a calculation is a lot easier than finding and justifying all the numbers in the first place, and there are plenty of undergrads ;-). We might not be able to replicate every lab experiment, but re-checking published calculations is well within the realm of possibility.
The paper in question:
Costanza, R., d’Arge, R., de Groot, R., Farber, S., Grasso, M., Hannon, B., Limburg, K., Naeem, S., O’Neill, R., Paruelo, J., Raskin, R., Sutton, P., & van den Belt, M. (1997). The value of the world’s ecosystem services and natural capital Nature, 387 (6630), 253-260 DOI: 10.1038/387253a0
The supplementary information no longer seems to be on the Nature website, but there’s a copy linked from this Duke University page.