An occasional blog by Alastair Otter, writer, editor, hacker, all round geek. Topics are likely to include media, data journalism, gadgets and software. Follow me on Twitter for more: @alastairotter

Media Picks #3

With the end of the year in sight retrospectives and predictions abound, headed up by Nieman Labs’ Predictions for Journalism for 2015. If you haven’t already checked this out it is definitely worth putting on your holiday reading list.

Subscribe: If you’re keen to join the exclusive Media Picks club you can subscribe to the weekly newsletter here or by using the form at the end of this post.

Smart (news) filters on the rise
The old challenge was often phrased as getting it first or getting it right, but in the knowledge-first era journalists face a new mandate — getting it to make sense.

The fall and rise of the news bundle
The notifications from news apps that make it onto that lock screen are in prime position to capture attention. The lock screen is the new bundle.

One of the big media moves locally over the past month has been the appointment of Andrew Trench as the News24 editor. Trench has a reputation for pushing the boundaries of storytelling as well as having an appetite for finding new ways of doing news. In From print trenches to digital forefront Trench talks about his plans for SA’s largest news site.

Mobile tools for data collection
Mobile is big news in the news industry as publishers do their best to get their content in front of an increasingly mobile readership. But what of using mobile devices as news and data collection tools? Joachim Mangilima has put together a great introductory list of mobile data collection tools.

There’s never been a better time to run a niche media business
It’s easy to look around the media houses making waves (think Buzzfeed, Vox and company) and think that the news industry is all about the big players while the small fry get, well, fried. There is some indication, however, that that’s not the case and there is a growing opportunity for niche players to thrive in the market.

And what about the nano-publishers?
Felix Salmon has been one of the voices championing the role of niche publications in the face of the media giants. In an intriguing piece on medium Salmon talks to one of the current leading examples of niche publishing, Ben Thompson, about the role of the nano-publisher.

10 funding options for journalism that blossomed in 2014
There’s no shortage of great journalistic content being produced right now. From video interviews to longform writing to entirely reimagined storytelling vehicles, journalistic creativity and heft are at a high. But how do you pay for it?

The beginning of the end of Facebook’s traffic engine
In a provocative piece Felix Salmon argues that the age of Facebook as a major driver of news traffic is starting to unravel. “The losers are going to be external websites who have become reliant on the Facebook traffic firehose. That traffic is going to start falling, in 2015, for the first time.”

VIDEO: Jeff Jarvis on journalism as a service
You may have heard that journalism professor Jeff Jarvis is publishing his latest book, Geeks Bearing Gifts, for free on Medium. It’s worth a read but if you want a short(ish) summary of some of the concepts in the book it’s worth watching this interview with Jarvis in which he expands on some of his ideas.

Tool of the week
Kimono Labs
There are a fair number of web scraping tools around but Kimono Labs’ tool is one of the easiest to use. It’s literally click-and-scrape: load up a website with the data you want, select the elements you want too scrape, and save. The scraped data can be downloaded in a variety of formats, including an API if you want to continuously scrape the data as it gets updated.

And finally, How we used WhatsApp for storytelling
A German news organisation commemorates the 70th anniversary of the bombing of Heilbronn using WhatsApp, with great success.

This will be the final Media Picks newsletter of 2014. Enjoy the holidays and here’s to a profitable news new year.

Subscribe to the weekly Media Picks newsletter

Your email address:

Media Picks #2

Missed Media Picks #1? Check out the archives.

Podcasting is apparently back and there are dozens of excellent podcasts you ought to be listening to (this american life, serial, startup, radiolab, all songs consideredtwit) but now we have an excellent new home-grown podcast in Jonathan Ancer’s Extraordinary Lives. Check out the episodes on Thuli Madonsela and Arthur Chaskalson for starters.


Subscribe: If you’re keen to join the exclusive Media Picks club you can subscribe to the weekly newsletter here or by using the form at the end of this post.


From silos to aggregators

Publishers thought mobile devices might let them retake control of the distribution channel. But they haven’t had as much success as they’d hoped for driving users to their single-source apps. As the initial thrill of iPad apps wears off many are now doing revenue share deals with aggregators like Flipboard and News360. In part this is because the appetite for standalone news apps is waning – readers want news from a range of sources – and in part because they are news organisations, not tech companies. Rather leave the heavy tech to the specialists.


Someone else saying that news organisations should focus on news rather than building out technology portfolios is Guardian CEO Andrew Miller:

Guardian CEO: “The idea we will survive by becoming a technology company is garbage”

“Why do so many of these technology businesses want to become media companies? We are a media company, we are about breaking news and strong stories and holding people to account and the idea that we will survive by becoming a technology company is garbage.”


Which is not to say the Guardian doesn’t invest heavily in technology, it does, but mostly to make its journalists more informed and efficient.

Behind the scenes with the Guardian’s analytics tool

Ophan is the Guardian’s in-house analytics system and Chris Moran gives a fascinating tour of the scope of this homegrown tool and the motivation behind it. “We don’t know enough about the internet as a publishing medium. We know everything about print, pretty much, there’s not many tricks left in the bag, we’ve done it for 200 years and we’re used to it. But the internet’s changing all the time, as much as anything else.”


And then there are those going it alone and (some are) living the dream:

Those bloggers earn how much?

A group of bloggers open up about the money they earn from blogging. In some cases it’s jaw-droppingly impressive. Who said it’s not possible to be a one person media company?

Someone else building a mini media empire is Ben Thompson who is proving Kevin Kelly’s maxim that it’s possible to make a decent living with just 1000 fans.


Technology trends journalists should watch in 2015

When it comes to predicting the future of news Amy Webb is always worth a listen. Her company just released an annual report looking at trends for 2015. This is a great summary of the highlights. If that’s not enough the full report can be found here.


Surviving media evolution

This is what happens when you get some of the smartest media people onto a stage together. It’s a little stream of consciousness like but still some good insights from the likes of Ben Hu and Frédéric Filloux.


The toolbox


Seeing dozens of things on the internet you want to read but you don’t have time right now? Save it to Pocket and catch up later. Pocket strips the cruft from articles and gives you just the text. It’s also an easy way to collect and organise articles. We use it in our news-gathering process as a way to store articles for review and possible later use.


Site of the week just gets easier and easier to use. If you need to scrape data from a website then is a good starting point. And if you’re new to scraping then check out this piece for a good introduction to

Web scraping in under 60 seconds: the magic of


Subscribe to the weekly Media Picks newsletter

Your email address:


Media Picks #1

We’re nearing the end of 2014 at a rapid pace. So what better way to kick off the almost-holidays than with a new little side project? I’ve been failing dismally at maintaining my blog this past year so I’ve decided to get a head-start on 2015 with a project to round up the best media and journalism news of the week. I’ll be posting that here and in email newsletter form to my enormous subscriber base of three. Or was it four?

Subscribe: If you’re keen to join the exclusive Media Picks club you can subscribe to the weekly newsletter here or by using the form at the end of this post.

2015 feels to me to be shaping up to be a major year for journalism and media. The slow burn decline in print of the past few years is likely to turn to searing heat next year as digital, particularly mobile, takes centre stage.

Speaking of change here are a couple some thoughts on the changes that are affecting the journalism industry:


1- The big trends changing journalism

Jay Rosen has put together a list to the trends that are shaping the news industry, from social, to mobile, to analytics in the newsroom. Rosen’s list, drawn from his digital thinking class, is full of examples and well worth trawling through.

Related: Steve Buttry’s excellent list of revenue streams for news organisations.


2 – Media outlets turn to Tumblr to extend reach

We go where the people are. At least that’s the attitude some newsrooms are adopting with many of them turning to Tumblr as another outlet for their content. Naturally pictures are a big part of this move and many newsrooms are using Tumblr to showcase their data visualisations.


3 – The dark side of social sharing 

Dark social? Yes, it’s those readers that, annoyingly, insist on sharing our stories in ways we can’t see, like email, or WhatsApp or SMS. Is it important for publishers? The Guardian thinks so, primarily because it affects their ability to provide advertisers with quality data about their readers.

And, this just in: Chartbeat flips the switch on dark social and discovers a high proportion of dark social sharing is not just from SMS and email but also from mobile apps, particularly Facebook mobile. For a more in-depth analysis of dark social read Alexis Madrigal’s Dark social traffic in the mobile app era.


4 – What does the future of online commenting look like?

Online comments are the best and the worst of digital publishing. Equally hateful and informative, engaging and alienating, they embody the the best and the worst of human behaviour.  We can’t shut them down – we want readers to engage – and we can’t control them completely because we don’t have the resources. What to do?

RelatedComments under news articles: Is it time to switch them off?


5 – Change the culture of journalism, not just what it produces

Obvious but always good to see it stated so clearly.


The toolbox
The must-have tools for the modern journalist


Most reputable sites have RSS feeds. Take advantage of them with Feedly. Plug your favourite sites’ RSS feeds into Feedly and monitor them in one place. We use Feedly to source most of our news. (We’re not paid to say that).


Site of the week

Anything and everything you’ve ever wanted to ask about being a mobile journalism is answered here. Nick’s site is chock-full of tips from recording interviews to doing mobile phone live broadcasts, to picking the best microphones, headphones and USB devices for your phone. A great resource.


And finally …

The modern beauty of 19th century visualisations

Cast your eye over these beautiful visualisations. They’re reproductions of visualisations from an 1870s statistical atlas.



Subscribe to the weekly Media Picks newsletter

Your email address:


Hacking education data: a school finder

Earlier this year, as part of another project, I downloaded the South Africa education department’s list of schools. At the same time I had a copy of the 2013 matric results for all public schools (at that stage in PDF format). It occurred to me it might be worth trying to put the two together to create some sort of schools information tool.

Of course, like most things, that is easier said than done. The first major hurdle was getting the matric results out of hundreds of heavily designed PDF pages. Pretty much nothing was up to the task although Tabula did the best of the lot. Eventually I gave up the task and went straight to the education department. It took a day or two but eventually I had a copy of the 2013 results.

With two handy spreadsheets in hand it seemed a simple task to join them together (they both had school IDs which ought to match). Again, not so simple. There were lost of discrepancies between the two lists and they were in serious need of cleaning (OpenRefine to the rescue).

schoolfinderBecause I was doing this as a side project I spread the work over a couple of weeks, picking it up when I had some spare time. Verifying the joined tables was pretty time consuming, as was getting the data into a format that I could insert into a MySQL database. But eventually it was done and I could start building a front end to it.

The current version is a relatively simple School Finder. It includes all of the schools info I could get my hands on, for both public and private schools. For now it only includes the matric results for public schools. I am now looking to see how I can add in private schools data.

The final product is built using PHP, MySQL, JQuery, and Google’s maps. The maps are static for now because I had some issues using the interactive version with a large number of data points.

Instagramming the city

As part of some testing earlier today I was playing around with Instagram. I’ve never been a heavy user of Instagram but I was looking setting up an account for our newsroom. Along the way I got distracted by some of the great pictures I found. To test Instagram’s embed capabilities I put together a small gallery of photos of Johannesburg, one of my favourite photographic subject matters. I even indulgently included one my own photos. Click on my name if you want to follow me.

Continue reading

Easy map visualisations with Mapstarter

In the world of data visualisation d3.js is the toolset of choice for most interactive journalists. But D3 also comes with a steep learning curve that makes it relatively inaccessible to the average journalist with just a small amount of coding experience.

I’ve done some very basic D3 stuff in the past but it took absolutely ages to get it working properly and I broke the visualisations more often than I improved them. And I certainly never got close to making anything that even resembled a map visualisation in all the time I spent trying to learn D3. So Mapstarter is something of a blessing.

Mapstarter makes it simple to create a basic interactive map from shapefiles as well as GeoJSON and TopoJSON files. And even if you do know how to programme your own D3 map, Mapstarter speeds up the time to get a map from concept to reality.

I heard about Mapstarter a couple of days ago (ht: @siyafrica) and decided to give it a spin. I downloaded a shape file from the South African Demarcation Board website (in this case an election ward map for Johannesburg) and 10 minutes later I had a functioning map of the number of registered voters in each ward.

Mapstarter is literally that, a “starter”. Once you have the map created you can tweak the information and styles and, if you know some programming, you can build out even more impressive map visualisations.

The original map I made popped up the number of voters in each ward when you hovered over it but it was extremely basic. So I opened up the shape file database file in LibreOffice, added another column for label text and rebuilt the map. This version is still very basic but at least you can tell what the numbers represent.

Once the map is created you can download it as a SVG or image file which you could use in Illustrator as the basis for an illustration, or you can download the code and include that in your website. That’s what I’ve done above.


Learning curves, and graphs

I’m rather pleased with this interactive graph, not because it’s particularly good but because it’s my first foray into D3.js.

Over the past few months I’ve been doing a lot of data-based work and while there are some excellent data visualisation tools available on the web, I’m always frustrated because I don’t have complete control over the final product. So this weekend I set out to learn D3.js. For those not aware of this D3 is a Javascript library similar to JQuery but specialising in data-driven visual effects.

So, after spending most of my long weekend glued to my computer I can finally show off something that I wrote from scratch. In all honesty it’s not particularly wonderful and not as attractive as some of the other graphs I’ve done using free tools on the web but it is a start.

If for some reason you decide to try and learn D3.js, do yourself a favour and start with this excellent tutorial by Scott Murray: Interactive Data Visualization. If it wasn’t for this excellent introduction I’m pretty sure I wouldn’t be anywhere close to where I have managed to get to.

Nkandla vs. Oscar Pistorius in tweets

Earlier this week I put together a small set of scripts to track the amount of attention the Oscar Pistorius trial was getting in Twitter. With not only local but also international audiences keen to follow the murder trial of the celebrity athlete it wasn’t surprising that Oscar was big on the social network. As you can see from this the Twitter activity around the Oscar trial regularly topped 2,000 tweets an hour while court was in session and on Wednesday peaked at just under 4,000 an hour.

So, when on Wednesday the Public Protector Thuli Madonsela finally released her investigative report into Jacob Zuma’s Nkandla residence I decided to do the same to see if people cared as much about the allegations against the president as they did about the gory details of a celebrity murder trial.

The good news is that they do, and significantly so (hover over chart for details):

Using the exact same method as I used with the Oscar Pistorius trial, I tracked mentions of Nkandla on Twitter from 10am on the morning of the announcement until 1pm the following day. As we can see there were just over 500 Nkandla-related tweets an hour at 12pm, half an hour before the announcement, but that quickly spiked to just over 6,000 an hour by 2pm, just over an hour into the announcement.

Oscar Pistorius: day 11 morning session in words

We’re into the 11th day of the Oscar Pistorius murder trial and during a particularly long, drawn out pre-lunch session I was messing around with the a large collection of tweets from the morning’s session that I had collected. What to do with almost 10,000 tweets but make a word cloud? So here it is: a quick wordcloud generated from 9,996 tweets over the course of the morning’s session.


It’s worth mentioning that the wordcloud was created with the excellent Wordle wordcloud generator. The tweets that were used in this were collected automatically using a series of scripts I wrote over the weekend. But more on that later.