Social media needs to get closer to social science

Social media should be the perfect thing to research.

For the first time in history, there is a record of what vast numbers of people say to each other together with a record of where they say it. We don’t have to ask people questions about what they do on social media, we can read their answers right there in front of us.

This is a big deal for all sorts of reasons (I’m going to focus on the commercial ones). In theory, our clients can see the results of their competitors’ campaigns before they sign off their own (normally they’d never get to see that research). They can see what sort of messages spread, where they go, and which influential people pass them on. They can see which ones get most attention from their customers. And which of those are close to purchase. They don’t have to rely on post-rationalised survey responses. AND… the data’s pretty much free.

Sadly, in my experience, it doesn’t quite work out like that. Here are my thoughts on why that is and what can be done about it.

Dealing with the data

In practice, the amount of data available is a blessing and a curse. There is so much that you can easily spend all your time trying to deal with it rather than analyzing it.

Nike, for example, was mentioned 645,000 times in the last month. If they want to compare themselves to adidas, they would have another 195,000 to deal with. Even Airwalk (a now sadly diminished brand of my youth) had 1,300 mentions.

And when you consider that for good analysis you need to classify and tag each item by medium (e.g. twitter or blog post), author, content, and relationship to other pieces of text, it is too big a job to do manually. You need an automated tool.

Now, this shouldn’t be a problem – after all, you’ve probably used an automated tool like Radian 6 or Sysmos to download the data in the first place. Typically these also come with a suite of tools that report the data as metrics like: “overall volume”, “share of conversation”, “sentiment”. They also have tools to identify “influential authors” and “hot conversations” so you can spend further time analyzing them.

This is all helpful. The metrics based on volume are simple and, once you have disambiguated any irrelevant content can be relied on. And they certainly do enable you to find nuggets of golden authors and conversations amongst the dross.

The difficultly though, is that you don’t know how the more interesting metrics were calculated. The influential authors look influential but how many equally good ones have been left out? What if one author is very influential within in a particular group but doesn’t have many public links to his name? Will he get picked up?

This causes me two problems. First, if I don’t know how something was calculated I don’t feel comfortable reporting it. Second, perhaps more seriously, it means I can’t change the influence model or the hot conversation model to take account of different market structures. This means I would have to analyse Obama’s social media success in the same way as Starbucks. Which sounds pretty daft to me.

Most of us are stuck monitoring

Roughly speaking, most social media measurement clients get:

-        A review of activity on their Facebook fan page – number of fans, number of posts, activity per post, most active participants,

-        Less detailed review of activity on competitors fan pages

-        Overall volume and rhythms of their category in the wider web – e.g. is the volume growing or falling; does chatter peak around a particular date?

-        Their share of the overall chatter, what their competitors get and roughly what subjects are associated with each brand

-        Success of any marketing initiatives based on the response on the response online

-        Which posts got the most attention – which means over time we can start to see simple relationships like “cocktail menus get more response that brand ads the Malibu Facebook page”

-        An indication of influential or hot areas (without justification or explanation)

In short – we can give clients really good descriptions of what is happening but we can’t really explain why. To use a physics analogy, we’re like Tycho Brahe (http://en NULL.wikipedia NULL.org/wiki/Tycho_Brahe) making measurements before Copernicus comes along and figures out that the earth is actually going round the sun.

Fortunately this data is so new, and the analysis is so far in advance of what has been available from other media, that clients still seem to be happy. I expect, and secretly hope, that this will not remain the case for long.

So, what should we be doing?

While monitoring is nice, being able to make accurate predictions about responses to marketing would have real commercial value.

We can already do a bit of this – e.g. pointing out that cocktail recipes generate more engagement than brand posts on Facebook – but it is at quite a low level. It would be a lot more useful if we were able to tell a brand how messages and ideas move around their market.

One big question in this vein, is whether conversation is thin and dispersed, that is, made up of lots of short shallow conversations between individuals; or whether it is lumpy and grouped together, involving in-depth group discussions whose conclusions spread across the population.

It seems to me that the two structures require completely different strategic responses. If it is thin, you need a message that lots of people find easy to remember, easy to explain, and which gives them a good reason to pass it on. If it is lumpy, you need a much longer message that will convince the most engaged participants in each group. And yet, our current tools give no way to choose between them.

To answer this question I think we should draw on insights from more mature areas of social science. It strikes me that relevant areas include Memetics (http://en NULL.wikipedia NULL.org/wiki/Memetics), agent based network modeling (http://en NULL.wikipedia NULL.org/wiki/Agent-based_model), and behavioral economics (http://en NULL.wikipedia NULL.org/wiki/Behavioral_economics), although I am sure there are many other possibilities.

By borrowing and applying and by developing our own theories, we can build a body of thinking to help us interpret our observations. We could do this separately but it would probably get done a lot quicker if we cooperated. Idealistic but lets see…

The other thing we need is raw data, not data that’s been classified by a proprietary tool that won’t show its workings. So, in another burst of idealism, why don’t we build an open source one? It would be a gift to every researcher, thinker, consultant, and social media practitioner in the world.

So that’s my proposal. Let’s stop monitoring. Let’s talk to academics. Let’s make our own tools. And lets figure out how on earth people are using social media.

P.S. I’ve written this post from a commercial perspective as this is a commercial blog. If anything though, the question of who gets to interpret discourse is even more pressing for the public sphere. That post can wait for another day though…