SxSW 2017: Looking back on 20 years of data

One of the best tips for a successful SXSW visit is to attend the sessions which lie outside your field of expertise. However, as a data-nerd, when about.com presents an analysis of 20 years of data, I couldn’t miss the opportunity to be part of it.

About.com. Maybe not the site most frequently used in Europe, but as a combination between Google and Wikipedia, in the United States it is not small fry. It was set up a year before Google and, with a history spanning 20 years, now has 3.5 million content pages. In short, it is one of the dinosaurs of the internet.

The heartbeat of the internet

The most remarkable thing that Dr.Jon Roberts, Chief Innovation Officer at about.com, tells us is that during the past 20 years, nothing has actually radically changed in our search behaviour. Of course, there have been technical innovations, changing trends and historical events. And, of course, the answers to some questions are now different. However, our questions, and therefore our need for information, are basically very predictable.

In a fascinating graph, Roberts showed the categorised growth in the content pages of about.com over the past two decades. The first remarkable fact is the jagged edge. Every winter season, there is a peak with a chunk missing – the month of December. It shows a rhythm, the heartbeat of the internet.

Predictable capacity

Within this heartbeat, you basically see the same thing year after year. Roberts explains this on the basis of the interest measured with regard to searching under the word ‘flu’. The number of searches in the summer months is low, but increases as autumn approaches, decreases slightly during the festive season and then gradually decreases again. Year in, year out.

On the basis of these tremendous datasets, patterns can therefore be derived. The trick to good data analysis is not so much pointing out these patterns, but finding deviations to them. You make a prediction of what a trendline will be like, and the deviations from that trendline are precisely the most interesting. Roberts uses the category of ‘health’ again as an example:

A vital event in the data is, of course, 9/11. An obvious effect was the increased number of searches regarding terrorism. However, an unusual second deviation from the data concerned weight loss. As if the whole of America wanted to be able to race down the stairs quicker.

At Christmas time, there is generally no traffic to health pages. However, after the election of Trump on 9 November last year, there was a zero line not previously shown. Conclusion? America had a headache that lasted three days.

Demographics show patterns which are the most interesting. For instance, Roberts shows the difference between age groups by means of the analysis that young people search more often for eating disorders and STDs, while people in their thirties search for pregnancy and miscarriages.

Young women are more interested in a weekend in Paris than older women. On the other hand, men of all ages show an equally low interest.

Most sought-after takeaway?

It was an enjoyable talk, but not the height of inspiration. Yet, during the Q&A, Roberts ended on an impressive note. He zoomed in on segmentation: good predictions can be made from aggregated data, but the more you look in detail, the more erratic it becomes.

You can recognise and help segments, but as for an individual? Well, an individual is practically impossible to predict.

This forms a good bridge to personalisation routes because, of course, a one-to-one personal approach sounds super sexy. However, it is also exponentially complicated and expensive. The advice is to think back on the repeated trends and, from this starting point, anticipate deviations from these patterns. You’ll still be aiming to deliver the right message at the right time, but by focusing on segments, as opposed to individuals, you’ll be more efficient and cost effective.

And that’s the great thing about SXSW - from an unexpected angle, you are sent in exactly the right direction.

Any questions?

If you need more information or have any questions just get in touch and we'd be happy to answer them for you.