Broadly speaking, and within the context of gathering qualitative consumer insights, online communities fall into two categories of interest;

1. Existing communities: virtual spaces where people with shared interests have gathered

2. Manufactured communities: virtual spaces where people who have been paid, or given some other incentive to offer their opinion come together

Are online communities, as described above, the goldmines for insight they’re being sold as?

Here are my thoughts;

Existing communities

These are communities that have naturally evolved to address member based interests and needs. Sport, parenting, cooking, fashion, music, social media etc – all have online communities that congregate in various virtual spaces across social media land (eg MSN or Yahoo groups, Facebook, blogs etc).

Theoretically at least, existing communities are a great resource for qualitative researchers. ‘Real’ conversations that often happen in relatively real time, minus any research effect. Insight heaven!

But what about in practice? Three key issues come to mind;

1. Sample

My old favourite, sample. Who are the community members? Do they represent the target market? There’s no way of telling.

2. Access

Many existing communities, and possibly the most interesting ones, are private; unsearchable for a start, but also, locked behind membership and passwords. If you can’t see them or hear them, it’s going to be difficult to glean any insights!

3. Professional ethics

Without the context of the market research ‘deal’, where critically, research participants are aware of, understand, and agree to participate in the research process, how should one proceed?

Should the researcher disclose their market research agenda? Is it unethical not to do so?

And if they do disclose their role as research participant/observer, how will the community respond? What effect will the researcher’s presence have on the community’s ecosystem and/or shape of the discussion? These unknowns make analysis a risky business.

Manufactured communities

Many researchers call them communities. They sell them as communities. Quite frankly, I’m not sure that manufactured communities (manufactured for the purposes of market research) should actually be called communities at all. Rather liberal use of the word ‘community’ in my opinion. A more accurate description would be a ‘purpose built environment’.

But notwithstanding the misnomer, from where I sit, manufactured communities are simply an inefficient, high cost, low return version of an online bulletin board focus group.

Here are my questions about manufactured communities;

What’s the cost of manufacturing, hosting, nurturing, monitoring and maintaining them to gather qualitative insights?

Where does analysis begin? Where does it end? What kind of questions are you trying to answer anyway?

You want consumer insights?

I can give you a thousand insights.

Here’s a conversation we once had with a potential client.

Potential Client: “We need insights!”

Us: “Tell us more” (spot the qualitative researchers).

Potential Client, in a louder voice: “We need insights!”

Us: “We can give you a thousand insights”

Potential Client, now smiling broadly and dancing with delight: “Wonderful!”

Us: “What are you going to do with them?”

Potential Client, somewhat soberly: “Oh.”

There’s a happy ending: the Potential Client became an Actual Client and over time, we gleaned the specific kind of insights they needed in a practical sense. Insights with focus and bite.

I tell this story because it highlights one of the issues I have with online communities for harnessing qualitative insights (they’re all the rage, you know) (both communities and insights that is).

My issue is around efficiency. Are online communities the most efficient way to get insights with focus and bite?

Of course, the answer is;

It depends! 

But for the most part, probably not.

Hang on to that cliff; more on this shortly.

Sentiment is the very essence of what we’re trying to understand through market research. And this is something that social media monitors (SMMs) don’t gauge very well.

Although automated sentiment analysis is often sold with the SMM package, there are two things about it that trouble me;

1.    Accuracy

2.    Specificity

Accuracy

There seems to be considerable scope for error in the labelling.

For example, how would automated sentiment analysis label a statement such as “F&*#ing brilliant!”?

Depending on the context, this statement could be;

1.    Dripping with irony

2.    An exclamation of genuine excitement and joy

3.    A description of a high wattage light bulb

So, would it be labelled as negative, positive or neutral? 

Notably, some SMMs claim to be contextually savvy, and that they can identify positive, negative or neutral sentiment with 90% accuracy (is 90% good enough?).

BUT…

Specificity

BUT (note caps), even with 90% accuracy, these labels are still seriously wanting. They don’t provide me with information that’s of much use – if any – because they’re too vague.

It’s the finer points of “sentiment”; the despair, frustration, excitement, boredom, curiosity etc underlying the positive or negative sentiment labels that I’m interested in. This is the level of sentiment I need if I’m to understand what’s going on with any effect. And to get to this level of sentiment, I really need to dig a bit deeper.

Where’s the gold?

I need to dig deeper, but where do I begin? Back to my earlier points about the issues with SMM sample definition and skews; I don’t know where the real gold (vs fool’s gold) lies.

Without spending the time and effort to sort through each and every buzzversation (possibly reaching the millions?), I can’t distinguish between content of import and that of little consequence. I just don’t know where to drill deeper in a meaningful, robust kind of way.

So it’s virtually back to square one.

Itching

SMMs are an exciting idea and I’m itching to find a way to use them.

From a PR or customer service point of view, I imagine they’re worth their weight in (real) gold.

But in my qualitative market research business, I’m not sure of how to use them with either confidence or pragmatic effect.

Sentiment aside, the sample scope/limitations and the unknown skews preclude the output from forming anything approximating a solid foundation for analysis. Bit of an issue for me.

Social media monitors (SMMs) trawl the web to find mentions of your brand or what ever it is that you’re interested in monitoring.

There are many SMM products and services available: some free, some you pay for.

Here’s a very basic example;
http://www.whostalkin.com/

If you type in the name of a brand or topic of interest, you’ll get an idea of the kind of information SMMs return.

Depending on the level of sophistication built into the SMM you use, you can refine your search with key words, run analytics, see where the buzz is happening etc.

There’s a lot of hype around SMMs. Not surprising really. The idea – getting feedback on the cyber-buzz around your brand, product or service – is timely and sounds quite marvelous!

Kind of. Until you think about it a bit more. Which I have. And wearing my qualitative researcher’s hat, SMMs actually fail in two important ways;

1. Sample definition
2. Sentiment

Sample definition

What constitutes a SMM sample? In a nutshell, a SMM sample comprises the searchable/findable content sourced from various online channels. That’s as precise as you can get really. The truth is, you just can’t know who’s represented (or not) within that content.

For example, SMMs can’t identify and screen out marketing blogs, websites or chatter. This means that SMMs don’t distinguish between content generated by marketing folk and content generated by non-marketing folk.

And let’s face it, quite a lot (most?) of the brand chatter out there is actually generated, nurtured and sent bouncing around the interwebs by marketing folk. People like us. The kind of people we try very hard to screen out of market research samples.

Also worth noting is that SMMs can’t distinguish between content generated by core customers, infrequent customers or non-customers. This means that all customer/brand relationship variations are automatically given the same share of voice and weight in the analysis.

Another factor to consider is that the sample will be skewed. And while a sample skew, in itself, is not necessarily a problem, it’s certainly a problem when you don’t know how it’s skewed. Which is the case here.

Without being able to define the sample, and without knowing how the sample is skewed, there’s no foundation or context for meaningful content analysis.

Next time, I’ll take a look at sentiment…

I, probably like many others who work in the marketing industry, am excited, but at the same time bewildered, by the online (r)evolution.

It’s not just the pace and possibilities of the Web 2.0 dynamic that intrigue me; it’s also trying to understand what it all means for the way I work. What does the online (r)evolution mean for the market research process and the research output?

But while I’m still struggling to define the precise (or even imprecise) nature, scope and impact of the changes we’re seeing/will see, one thing’s for sure; the amount of information available to me is growing. Exponentially.

As a market researcher, this is both, at once, a beautiful and terrifying thing.

It’s a beautiful thing, because you can never have enough information. In fact, information to a researcher is like mud to a pig.

But it’s terrifying too, because as a market researcher, my key task is to create, harness and manage information; to give it pragmatic wings.

How on earth am I going to do this in an ever deepening sea of what feels like, for the most part, un-coded, un-weighted data?

My mind has been thrown into a veritable swivet…

Of course, the principle that steers all good research remains the same; in approaching the un-ordered online research-scape with any hope of getting anything useful from it, you need to be armed with good questions. I’ve already done the good question post, so let’s assume the questions are sorted and the task at hand is to go about answering them.

Which online services and tools will get you the best return for your research efforts?

Notwithstanding that in practice, specific objectives will guide the choice of any particular research methodology, I do have some general thoughts about some of the Web 2.0 research tools I find increasingly at my disposal.

Stay tuned…

As you all probably know, I have a ‘thing’ for questions; love a good one, loathe a bad one.

Why? Because within a research context, asking good questions is absolutely, fundamentally, and without doubt, a prerequisite for getting good (ie useful) information.

And as we all try to acclimatise to the great unwashed world of Web 2.0 metrics and mumblings, it’s becoming increasingly obvious that asking good questions will be one of the greatest challenges – but will also return the greatest rewards – for marketers and market research consultants alike.

How sharp, how ‘good’ are your questions?

More on this to come…