|
May 2001 Volume 2.07
The Benefits of CVA (Part Two)
by Rodger Gallagher
Running The Survey
 If
you are going to make business decisions based on survey results
then the base data needs to be sound. People often regard fieldwork
as a commodity and select the lowest priced supplier. But fieldwork
quality varies immensely and it is critical to have high quality
fieldwork to ensure that the base data is sound. To get this we
need a survey that has both high precision and high accuracy.
"It is very important to use sample sizes that are large enough
to deliver research results with a low margin of error". Most people
would accept this as a true statement and one of the most important
factors to consider when running a survey. Sample size is put under
the microscope because it is easy to measure, but what people often
mean when they talk about sample size is the number of completed
responses to the survey. It is this that determines the precision
of the research (what margin of error will we tolerate?) But what
is even more important, but often spoken of in hushed tones, are
sampling coverage and non-sampling error.
The right coverage
By sampling coverage, we mean both that the sample we have drawn
reflects the same population we are interested in, and that the
sample has been drawn randomly so a complete range of views in the
population is represented. This is important as it determines the
accuracy of the research. This is especially so in CVA research
where we are interested primarily in the views of decision makers.
So first we need to ensure that we have the right population to
draw the sample from. This needs to be defined and specified in
the Developing The Research step. For example, in business-to-business
research we may wish to only include customers with either current
or potential high sales revenue in the research. And which people
in each business are the decision makers that we need to contact?
If named decision makers are available then that increases the chance
of contacting the right person and obtaining their views. Once the
population is defined then a probability sample must be drawn from
the entire population using a random sampling technique. This needs
to be large enough in size to provide the target number of completed
survey responses.
But supplied lists of customers may contain stale information where
people have moved to other roles or companies. Or the best list
available may be one of users rather than decision makers. To overcome
this or when supplied contact names are not available, questions
in the survey must be used to ensure that only the views of the
defined population are captured by the research. So a combination
of careful sample selection and screening questions in the survey
is necessary to ensure that we have the right coverage.
The unspoken error
A simple way of reducing the cost of market research is to only
survey people who are easy to contact. Possible respondents are
contacted once only using as many contact names as needed, until
the desired number of completed responses is obtained. This means
that views of people who were not available for one reason or another
on the initial contact are excluded from the survey. In surveys
of consumers it is usually easier to contact people who are unemployed
or retired. If a broader mix of coverage is required than this,
then research results based on these responses may well be wrong.
If any part of the target population is inaccessible, and therefore
not available for surveying, then this opens the potential for biased
statistics. Completed survey responses determined by convenience/
accessibility/ availability introduces the potential for unknown
and unknowable biases in statistics and does not enable their precision
to be measured.
To ensure that this type of bias is minimised it is critical to
measure the response rate. This is the percentage for the number
of responses, divided by the number of valid contacts made:
With well managed research, and if respondents are interested in
the subject, response rates can go as high as 70% but responses
of 40% are more typical. To achieve high response rates it is important
to use reminders for postal and Web based surveys. For telephone
surveys, a number of follow-up calls must be made across a number
of days, at different times of the day to improve the contact rate,
with appointments made for interviews at times that suit the person
being surveyed. If this is not done the number of 'no replies' or
'voice mail' contacts will bias the data collected towards the views
of people who are easy to contact.
By applying standards to the way the population is specified, the
sample is drawn, and the fieldwork is conducted, the CVA approach
ensures that data to be used for business decision-making is sound.
BELIEVING THE ADVERTISING
It was just before lunch and I was in the soup aisle of the supermarket.
I spotted a can of Tomato and Bacon soup with an attractive label
showing little bits of bacon floating on the soup. So off to the
checkout it went and then home to prepare a culinary delight with
a trusty can opener. After carefully following the instructions
I ladled the soup into a bowl. But it didn't look anything like
the bowl of soup on the label of the can. What had I done wrong?
Perhaps I had opened the can clockwise instead of anti clockwise?
Come to think of it, bacon would sink to the bottom of a bowl of
soup anyway. Well anyway the soup did taste of bacon and I did find
a few bits of the bacon on the bottom of the bowl.
But what was the photo on the can's label meant to depict? Well
I didn't find that out by reading the label, but what I did find
in letters one millimetre high were the two words "serving suggestion".
It didn't say what it was a serving suggestion for. I rang the 800
number on the label to ask what the photo was meant to depict, and
twice received the message "all of our operators are busy". Perhaps
too many other people were ringing after lunch to find out what
the photo was of? On calling back a third time I finally got through.
The person did not have a can label to look at but thought it might
be squares of toast or croutons in the soup. Since when have croutons
looked like pieces of bacon that float? My complaint was to be passed
on to the product manager. I suspect that it wasn't even recorded.
Is it any wonder then that people are very sceptical when it comes
to believing the claims made in advertisements? They prefer instead
to believe what their friends, relatives, and work mates tell them
about products and services. Well I won't be recommending a particular
brand of Tomato and Bacon soup to anyone. But if you are after a
can of soup I do recommend, then try Campbell's Homestyle Pea and
Ham. The contents look like the photo on the label and it tastes
great.
Regards,
Rodger Gallagher
|
