Focus Groups - To View Or Not To View ?

Whilst our research agency Divine Market Reserach www.divineresearch.com
is in the business of moderating groups it’s not so long ago that many of us were at the vantage point behind the glass. As clients and planners we used to watch people doing what we do now, and often not envy those doing our current job. It’s from this dual perspective that we feel marginally qualified to offer some observations and thoughts about the process of watching research groups.

It seems that the hiring of a viewing facility is rather a luxury in straitened times. So such moments are, and should be treated, as fairly precious quality time with the people who matter most (by this we mean customers).

But our experience and that of our colleagues elsewhere suggests that this is not always the case and that the opportunity to see and hear customers sharing their views is sometimes ignored or misused. So we thought it worth highlighting some specific examples of less-than-ideal approaches to watching research groups, the particular issues these expose and make some suggestions as to how to get the most out of the experience.

But first let’s think about why one might watch research groups.

There are a number of reasons to watch groups.

1.      To check that the process is being conducted properly and that the moderator is doing a good job, getting everything out of the research that needs to be covered. This is an understandable objective if a research company is new to the business or if there have been any areas of concern in the set up of the project but if a good relationship exists then this should not be a reason to come to watch.

2.     Out of interest. Sometimes clients just want to see the whites of their consumers’ eyes, hear them speak, get a glimpse of their world. This is perhaps the best reason for watching research groups. It doesn’t require attendance at every group in the project, it doesn’t require analysis of the learnings (which is what the research is being paid to provide), it should be an interesting part of the client’s day and give them some textural examples to use in their course of their work. But attendance at one or two groups should never be seen as more than a taster of the project as a whole. It might prompt questions to ask at the debrief, based on experience of just a few of the groups, but it would be dangerous to base any conclusions on what has been seen. However vociferously and unanimously an opinion was expressed in one group, if the other 7 demonstrated a different way of thinking then the viewed response must be seen in that context.

3.     To be able to report back to colleagues what has happened, perhaps to faciliate some premature decision making. This is not a reason to attend that many researchers would endorse. Unless the client viewer is going to come to all the groups, then go away and listen to/watch all the recordings again, analyse, pull apart and put together again all that was said, discuss extensively with colleagues, look for the significance of what wasn’t said as well as what was, review non-verbal cues i.e. do everything the researcher will be doing then there is a risk that the conclusions drawn from observation without analysis could be erroneous or misleading. And if the viewer is going to conduct analysis with rigour comparable to that of the researcher, it rather begs the question ‘why use a researcher?’

4.     To get a topline steer on what respondents think without any subsequent analysis, to be able to ask the right questions in the debrief and to be able to make observations to be challenged by analysis. Not a bad reason at all, as long as the distinction between observations and the insight born of analysis is appreciated.

5.      Because that’s what’s always done. This is fair enough, if habit has arisen for a good reason. See above…

Our conclusion from a brief survey of the reasons for viewing research groups (and we are sure to have missed some out) is that it is important to be clear why you’re viewing groups, what you plan to get out of it, what you need to do with what you see and whether this is a realistic and valid objective to have. Is viewing this group the best use of your time?

Now for some case studies that demonstrate problems that can arise when groups aren’t viewed in the right way.

Case study one:  Do you want questions or answers?

A moderator is conducting some script evaluation in a viewing facility in Moscow. All the scripts are attempting to communicate that a Product A manages to work quickly but also be gentle. A particular script provokes much discussion and it is clear to the moderator, from what the respondents are saying, that this script does successfully communicate both product benefits. Respondents are playing back this message in their own words. However, the client is concerned that the moderator has not explicitly asked the question ‘does this tell you that the product works quickly and gently?’The moderator has not asked this question because respondents have provided the answer spontaneously. She would learn nothing new by asking this question outright. But the client has seen a discussion guide which indicates that this question must be answered (note: answered) and has assumed this to mean that the question needs therefore to be asked. As we all know, the thing about research groups is that they don’t always exactly follow the discussion guide to the letter. Sometimes with some gentle nudges they pretty much moderate themselves, as a discussion ranges across all required topics and resolves all proposed questions without them being asked. Sometimes it is appropriate to re-order sections because the conversation more naturally flows that way. Sometimes a later question is answered much earlier and therefore doesn’t need to be covered again. 

What this illustrates: A couple of things. That discussion guides are guides not scripts. And that answers are more important than questions.  If the information required is elicited or offered in another way or at another time in the group it is still just as valid. Discussion guides are not checklists to be ticked off whilst watching a group, or certainly not in the order in which they are written. It’s fair enough to object if a whole area has been neglected but  as long as everything has been covered in some way then the group has been moderated properly. We would argue that discussion guides are of primary usefulness before the groups: they are the ideal opportunity to check that all objectives will be delivered, all areas will be covered, that everyone is happy with all techniques to be used, that the stimulus material can be effectively used.

Case study two: What you see is not all you get

We know a researcher who we rate exceptionally highly (we’re not telling you who he is though) but not all of his clients see him that way. They are uncertain about him because his groups sometimes seem a bit brief, not as full as they would imagine or expect. They have censured him for missing bits of the discussion guide, for finishing groups early. But his debriefs are brilliant – insightful, answering all the questions and responding to all the objectives in full and actionable ways. His groups might have appeared (wrongly) to be superficial but no-one could lay that accusation at his debriefs. And why is this? Because he’s so naturally intuitive and insightful, he can see and understand what lies behind what is said, he can get quickly to meaning without the need for slightly repetitive questions. There are a lot of researchers like him (we’d like to say we know some very close to home…) but what he doesn’t do is play to his audience, take any account of the people behind the glass and what they conventionally expect to see. He, rightly, sees qualitative research groups as being the means to answer questions but sees the answers as being of greater importance than the means. So he doesn’t perform, he does it his way.

What this illustrates: That sometimes the importance of how a group is moderated is over-estimated and the ‘performance’ of the researcher as moderator is judged separately from his/her overall performance on the project as a whole.

What can we learn from this? The way to judge is not on performance on the night but on what he/she comes back with, how well the debrief fulfils the objectives, how far it grows knowledge and understanding, how insightful and how useful it is. The end should very definitely justify the means.

Case study three: Not getting the ‘result’ desired

This is a less specific example as we have seen this so many times. We’re often called upon to research a number of ideas/concepts/routes alongside each other in order to determine which best fulfils the clients’ objectives for it. Although the term is pejorative it is essentially a beauty parade. The problem arises when there is not a clear winner, when several routes are perhaps deemed to be strong in different ways. Or something different wins in each group. Too often this is seen as a failure of research, that the researchers have not been able to identify the optimum idea. But sometimes there just isn’t a winner. This is no worse a result than the revelation of a clear direction. It might mean more work needs to be done but often the other learning and insight from the group is as directional and useful as the isolation of a lead route would be.

What this illustrates: That it is tempting to think that the answer will come from the obvious place whereas a more useful and interesting answer and direction might come from somewhere else. This also illustrates how research can be all too readily written off if it doesn’t provide an exact answer when in fact an exact answer may be elusive because none of the materials presented as stimulus contain the definitive answer.

What can we learn from this? To look at what a project as a whole reveals, to see how questions might be answered in unexpected ways and to understand that research might sometimes not be able to recognise, choose or highlight the optimum direction if the optimum direction has not yet been created. Sometimes research will point out where more work needs to be done, and good research will not shy away from doing this.

So, Divine’s top tips for getting the most out of viewed groups, in no particular order

1. Be clear why you’re going and what you want to get out of this? Is this feasible and is this fair?

2. Remember that what you see in one group is a fraction of what you’ll get at thedebrief, which is the result of a lot more analysis and insight. It’s more than the sum of the observation of every group conducted.

3. Don’t arrange to report back to colleagues immediately after the viewed groups: you’ll be giving them a small part of the story only

4. Don’t evaluate the moderator by how many questions he asks or how closely he sticks to the discussion guide.

5. And finally always try to remember that the discussion guide is a just that, not a script - some questions don’t need to be asked to be answered.

    

Halifax Tone It Down

It would appear that The Halifax have finally got wind of the fact that the we are having a bit of a tough time of it out in the real world. I say this because, as you'll have noticed, the advertising campaign featuring staff having a right old laugh at 'Radio Halifax' have finally been put to rest. Instead, the fun-loving financiers have now been brought together to give us an altogether more toned-down rendition of  'I'll be there'.

Now I'm tempted to suggest that if they really wanted to win hearts and minds, getting back behind the counter and putting the music careers on hold until the economy perks up a bit mightn't be a bad idea. The move to a more recessive staff presence in the new advertising (there's only 100 of them in the featured choir) may go some way to convince us that all is well again at The Halifax. It's possibly less convincing for the remaining staff that didn't make it into the choir - they're probably having to do double-shifts to cover for their singing chums who are now preparing for X Factor.

Whether or not you think the new advertising is an improvment on what has gone before, and I do, it is probably unfair to lay responsibility at the door of the incumbent advertising agency. One imagines that 'staff' remain a large element of the advertising brief.

There's a long history of staff and clients appearing in ads - who can forget Victor Kiam doing his best Leslie Nielsen impersonation. So let's not leap to judge. Instead, let's take the opportunity to enjoy this well-known advertising truism which sums up the issue of clients appearing in ads most eloquently,

“If the client moans and sighs,
make his logo twice the size.
If he still should prove refractory,
show a picture of his factory.
Only in the gravest cases,
should you show your client’s faces.”

Amen.

Slip-Sliding Away

In her recent slot with Mark Kermode, Lauren Laverne expressed genuine surprise to learn the origin of the word 'blockbuster' - just in case, that's when the cinema queue is so big it 'busts' the block. Now Mrs L is no irony-free zone but it was disconcerting that she might need a lesson on cinematic semantics. If she didn't know, then what hope for the rest of us ? And what other meanings are quietly slipping from our collective memory ?

Before we move on from 'blockbuster', it's probably only fair to acknowledge that it's a word that has been truly mangled over the years. No-one is more guilty of this (in my eyes at least) than 70's glam-gods, Sweet,

"Does anyone know the way, did we hear someone say
 We just haven't got a clue what to do
 Does anyone know the way, there's got to be a way
 To Block Buster! "

For goodness sake. Block (imaginary hphen) Buster ?

But back to the point. I recently found myself discussing with a group of marketers the origin of the expression 'lobby' (shame on you - look it up yourself). No fools this bunch, but as the conversation developed it became clear that we're dealing not only with a historical lexicon but also a growing number of new expressions. So perhaps it's just a case of survival of the fittest. In with the new and out with the old. Case in point - assuming that we have finite long-term memories, should one retain the origin of an old-school expression like 'bootleg' or let it slip and be replaced by something within the modern idiom like 'camming'.

Of course it's not an entirely rationale process but you get the point. Grapple with the burgeoning amount of contemporary in-bound or doggedly maintain a grip on the past ?

I have to say that as the senior moments come thicker and faster I know where my money is.

So perhaps LL wasn't being disengenuous, rather just letting-go gracefully ?

Did Barclays Bank On Boris ?

 

Recently a friend sent me this link to support his charity bike ride - don't worry this isn't a crafty piece of e-chugging (if you want to you can read more at http://uk.virginmoneygiving.com/fundraiser-web/fundraiser/showFundraiserProfilePage.action?userUrl=lindsaywilliams).

Beyond the ludicrous scale of his endeavours, what really caught my eye was the title.

'200 mile Boris Bike Challenge'

I like Boris Bikes. Aside from the fact that they encourage anyone with a debit card, a modicum of dutch-courage and delusions of immortality to weave around the capital, they add a bit of colour to the city. Barclays blue, to be precise.

Which brings me back to the 'Boris' in the title of the original email.

To be blunt, they're not Boris' bikes. They're Barclays'. I know because it says so on each and every one.

Now I'm not taking sides here. Both are brands in their own right, with clear positionings, one being banking, the other being barking. And of course it might well be that Barclays are perfectly happy with the status quo. My perception is that in these straitened times, if it weren't for their involvement we mightn't have the current scheme, so I am vaguely greatful. However you can't help but imagine that initial conversations between the development departments of TFL and Barclays focused largely on exposure to an appreciative London population and gave significantly less consideration to the image the Mayor pedalling his way on to the next PR gaffe.

Did the words 'Boris' and 'bike' crop-up in the same sentence during negotiations ? Or was the need for a plan 'B' even mooted in the cab (or tube) on the way back from the TFL offices ? Was Kelly Brook (remember her at the launch ?) meant to 'de-Boris' affairs  ? We'll probably never know.

If nothing else, it goes to show that commercial organisations need to give serious consideration to their brand values and those of potential sponsorship properties before forking over significant sums of cash. There are plenty of great examples of where a sponsor and a beneficiary have combined to create a much loved entity - look no further than the collaboration between The Southbank Centre and The Observer, which brought us the ever-popular Meltdown.

The birth of the 'Boris Bike' demonstrates the importance of three things. Firstly, the need to understand the associations that exist in the minds' of your target audience. Secondly, how critical it is to rigorously identify all the component elements of the partner brand. Thirdly, and probably most importantly, the acceptance that creating great sponsorships isn't a perfect science but one that more often than not, with reasonable planning can produce happy (if not entirely anticipated) accidents.

Something I suspect our mayor is acutely aware of.

Cultural Olympiad Who ?

"Dad, what's a Cultural Olympiad ?"

Now whilst I'm as big a fan of the Simpsons as the next man, I've never gone out of my way to play Homer to my thirteen year old daughter's Lisa. But as I struggled to replace the mental images of doughnuts with something more tangibly 'olympic' or 'cultural' it dawned on me that sugar-based pastry was about all I had to offer.

Now it's probably only fair to make a couple of things clear. Firstly, I have more than a passing association with the cultural sector, which makes my senior moment all the more inexcusable. Secondly, I'm well aware that said Olympiad has come into more than its fair share of bashing, a band-wagon that I've no wish to jump on. However, a straw-poll around the office quickly demonstrated that our collective knowledge placed all of us fairly and squarely in Springfield.

The best definition of an Olympiad we could find was this, "an Olympiad refers to a period beginning January 1 of a year in which the Summer Olympics are due to occur, and lasting four years". And if you go the 2012 website you'll find "Spread over four years, it is designed to give everyone in the UK a chance to be part of London 2012 and inspire creativity across all forms of culture, especially among young people."

All in accord there then.

Except for that word "spread".

Because to all outward appearances the Cultural Olympiad seems to have been spread fairly thinly to date, suggesting that 2012 is going to be fairly oozing (back to doughnuts again) with opportunities to participate. But appearances can be deceptive and for all I know the rest of the country is is being served a non-stop diet of cultural delights whilst I continue to grind away on the wrong side of the green door. Either way, it's time to find out.

Now I said that I have more than a passing association with the cultural sector and it so happens that one of our businesses, Muse, runs a Cultural Panel designed to capture the attitudes and opinions of culturaly engaged audiences. It's a significant enterprise, with over 30,000 attenders of galleries, museums and arts venues regularly offering their opinions on matters cultural. And in order to set the record straight we are going to use the Cultural Panel to establish what appetite there is for the Cultural Olympiad -  we'll measure the level to which our audience are aware of it, whether or not they are interested it and to what extent it appeals to them. We'll not only be able to establish what those levels are now but we'll also be able to track them in the run-up to the games themselves.

All of which I hope will, at best, evidence growing support for the Olympiad.

And at worst elevate me to mayor of Shelbyville.