As inexorable as the New Year is the set of retrospectives on the year before that will accompany the calendar change. A new short-hand for the digital age in this arena seems to be the top 10 blog posts of the year, based on the number of views. The Center for Global Development did it for their Health Policy Blog; Duncan Green of Oxfam has too, as has the DFID Research to Action site.
Should ODI do the same too? I don’t think so. I’m on record as being fairly sceptical about the use of page views and downloads on their own as indicators of success. Yes, these indicators provide useful data. However – as is being discussed in other circles relating to the rise of ‘big data’ – data does not equal information.
So, for my retrospective, I thought I’d do something a little different: the ODI online awards, 2012. Though the awards are based on data in our monitoring and evaluation dashboard, they also offer my views on why things were successful and any insights I can give into issues to take into account when reporting on success.
So, without further ado, here they are….
Most widely read new publication: Horizon 2025: creative destruction in the aid industry
This key report by Homi Kharas of Brookings and Andrew Rogerson of ODI on the aid industry made a large splash following its release in July. Mentions of it could be found in media outlets such as The Guardian, Chicago Tribune, the BBC and the Sydney Morning Herald; in blogs such as A View from The Cave, Devex and the Huffington Post; and by development organisations including the World Bank, World Vision and Save the Children. Over the months since, it has registered 6,622 downloads – putting it well ahead of the nearest competitor in terms of downloads, The euro zone crisis and developing countries, which registered 3,880.
An honourable mention should be reserved for another publication on the euro zone, released in November 2011. The euro zone crisis: risks for developing countries saw 6,807 downloads over the course of 2012. Had it been released a couple of months later it would have probably have pipped Horizon 2025 to the post in terms of downloads. Yet due to its late release in the year it wouldn’t even have made the 2011 top 10 – pointing to another vagary in top 10 lists and a good reason to not rely on downloads alone as indicators of success.
There was also lots of media traffic around the European Report on Development 2012, which ODI helped produce, but the report is hosted by the European Commission and we don’t therefore have download figures. More on this particular problem with download figures later on...
Most widely-viewed event series: The 'Calling...' event series run by the Humanitarian Policy Group
This event series used ODI’s video-conferencing facilities to bring together panels from Sudan, South Sudan and Afghanistan with panels in London. Events were broadcast around the world through video streaming and reached viewers around the globe. There were registrations to watch from as far afield as Zimbabwe, Iraq, Pakistan, the Philippines and the three countries themselves. Overall, we logged registrations from 48 different countries in all five continents – and figures actually viewing the content will be even higher, as catch-up video is available on our site and Youtube.
Most widely viewed infographic: 10 things to know about climate finance
OK, this is a bit of a cheat, as it was also our only infographic in 2012. But as infographics are the big thing at the moment on the internet, I wanted to highlight it here. But I also wanted to share it as this infographic tells a different story about success. The blog around the graphic received 2,017 views. The big story, however, is not how many times it was viewed but who it was viewed by. We happen to know that this infographic was shared among the negotiators on climate finance at the Doha summit. For me, and ODI, that is more important than thousands of downloads by the interested general public.
Most tweeted piece of content: Old puzzles, new pieces: development cooperation in tomorrow’s world
The ODI Centre for Aid and Public Expenditure conference saw 113 tweets mentioning the event page. However, this again has a lesson, and it is one that will become increasingly important to take on board as AltMetrics become more important to judging academic success. The issue is what makes up a figure – the 113 tweets about the conference include a number of tweets by ODI researchers and staff promoting the conference. Given this, a more likely winner for the most tweeted piece of content would be Inconvenient truths about corruption and development, a November 2012 blog written by Marta Foresti, which was tweeted about 99 times.
Most successful website: Post-2015.org
This website, curated by ODI to inform on the process to replace the Millennium Development Goals, was launched in April 2012. From a standing start it now receives over 10,000 views per month – 14,000 in November. That is the fastest growing research site I’ve seen in the six years I’ve been at ODI. Perhaps that is because it has, in those months, published 207 posts - that is over a post a day. This includes posts from people at numerous organisations working on the subject. This regular updating has made it a vital source for information in the sector, a fact also evidenced by the 719 followers it has quickly accrued on Twitter, including leading figures in the post-2015 debates.
Most widely read blog:A pragmatic guide to monitoring and evaluating research communications using digital tools
Registering 8,060 views over the course of 2012, this blog post I wrote for OnThinkTanks.org comes top of the list for blogs we have figures for. This points to another of the issues with top 10 lists, especially for ODI given our ‘being there’ digital communications strategy that encourages posting to large or sector-specific blogs wherever possible to reach a wider or more directly relevant audience. The problem is that if we post outside the ODI site then we can’t easily get figures - this blog is the exception as I can access OnThinkTanks.org statistics. So the most effective place to communicate is often also the least measurable. It is therefore likely that some of the blogs we’ve published in The Guardian, World Bank or any one of a number of platforms were actually more viewed. But we don’t know.
Most commented upon blog: ‘What works’? Systematic reviews in international development research and policy
As I said in my introduction, it isn’t just about views. The next award goes to the blog that managed to get most comments on the ODI site – a key indication that the blog has generated engagement and interest. This is no mean feat –around 70 percent of visitors to the ODI site come to research or look for jobs. They’re not necessarily in an interacting mood. Through strategic use of Twitter to engage people with an interest in the subject, the Secure Livelihoods Research Consortium team were able to interest Ben Goldacre, well known for writing ‘Bad Science’ and the discussions kicked off from there. The blog received 19 comments in all, good for the ODI site – though a drop in the ocean compared to the 600,000 comments the Guardian gets on their blogs every month. However at least every single comment on the ODI site was a valuable intervention – no trolls here. I hope it is ever thus.
Lifetime achievement award: How to write a communications strategy
Finally, a notable mention. For many years now, this 2003 online toolkit has been by far the most viewed piece of content. In 2012, the toolkit received 30,310 views and the downloadable version was accessed 8,410 times. Why is this still getting so much attention? One word: Google. The piece is often at the top or near the top for searches on how to write a communications strategy and the visits come as a result. This doesn’t necessarily mean much for ODI and its’ impact, however, as most of the visitors have very little interest in the rest of ODI’s work. But it does show yet another reason why just looking at page views can give you a very skewed idea of success, one that is not very useful to think tanks trying to inform and influence policy in the world.