9 issues to consider when deciding between Telephone or Door-to-door surveys in small markets?

The source for this post is the textbook Marketing Research by Aaker, Kumar and Day, supplemented by our own experience in the small markets of the Caribbean.

When trying to decide between a telephone or door-to-door (face-to-face) survey, these are some issues to consider:

1. turn-around time - Because it is faster to call someone than to drive or walk to them, the turn around time for a telephone survey is shorter than for a face-to-face survey. In our experience, 400 telephone interviews can easily be done in less than 80 man-hours. Not so with face-to-face interviews
2. length of the questionnaire - If the questionnaire is long or complex, it is often not desirable to conduct the interview over the phone. We have found that sometimes respondents are more willing to reveal certain personal information over the phone than in person and vice versa.
3. distance to travel to respondents. If respondents live far apart, a telephone survey might be more cost-effective because of the cost and time of travel.
4. level of supervision. It is easier (and more cost effective) to duly supervise interviewers from a centralized call area, than on the road. Mistakes are also easier caught and corrected, especially if data input happens at the same time, as we do at MarkStra.
5. accessibility to respondents. Not all people have a telephone; interviewers can often not gain access to gated residential communities or apartments; interviewer safety may preclude visits to certain neighborhoods or during certain times of the day. Other groups are systematically inaccessible because of lifestyle or unwillingness to cooperate. Young men are often not home. When they are not, they are likely to be engaging in a group activity and cannot participate, even when called on a mobile phone. Workers in hospitality and health care often work shifts and may not be accessible during "regular" interviewing hours. In Curacao, we find that European Dutch people are systematically unavailable during dinner hours when interviewers are most likely to call or visit.
6. product category. If the survey is about Ringtones, a phone is a pre-requisite. Hence a telephone survey is highly appropriate. If the survey is about a basic necessity (such as water or electricity), some respondents are likely to not have a phone, while their opinion is not only important, but possibly varies significantly from people with phones. If someone has an issue with the electricity company and has a phone, the road to a solution is probably much different than if he doesn't have a phone.
7. sampling method used. Random sampling can be more effectively executed with a telephone survey. It is easier to call and call-back the chosen respondents, than to drive to their house with the risk of not finding them home or the risk of them not qualifying for the survey (f.i. because they are not a user of the product).
8. the number of phones available at the call center
9. budget- total cost is determined among others by the issues above

I started writing this post to explain the limitations of telephone surveys, but have realized that face-to-face interviews have at least as many limitations. Point 5, not being able to enter gated communities and apartments and interviewer safety issues are especially telling.

In the Caribbean, ROI and budget issues are especially important.Telephone surveys probably fit the budget better. Don't dismiss them as being less-than-ideal. Door-to-door surveys have at least as many limitations.

Survey sample sizes in small markets

Earlier this week I had a FB discussion with a fellow researcher about appropriate sample sizes (steekproefgroottes, muestras) in market research. It prompted me to write a post I always want to write each time that discussion comes up in Curacao.

As I write, I realize that the discussion might arise from the different goals of business research versus scientific research. In business (and other organizations) information is gathered to assist in decision-making. It has to make business sense. Scientific research often merely seeks to describe a condition. The findings are not necessarily used to aid in decision-making.

The research field allows for all sample sizes. Researcher just have to indicate the accuracy of the findings.

The larger the sample size, the more accurate the result, i.e. the more likely it is that the result of the study is closer to the reality. What is the error?

Sample (rounded)	Error (rounded)
2.500	2%
1.000	3%
600	4%
400	5%
100	10%

This means that if you interview 400 people and 60% says they like your product, the reality is that between 55% and 65% like your product. If you interview 2.500 people, and 60% says they like your product, the real answer is likely to lie between 58% and 62%.

So, why would a market research not choose to work with the most accurate (large) sample size?

1. ROI reasons. Let’s say that a mass market research study with a sample size of 2.500 costs the same in Curacao, Holland and the USA, e.g. USD 10.000. All else equal, if the accurate answer on which you will base a decision yields you USD 1million extra profit in Holland, it will yield you USD 10.000 extra profit in Curacao – a BREAKEVEN -, just because our market is 100 times smaller (16 million vs 150.000). Why bother? What are the chances that you will sell this research proposal to your US boss, who is accustomed to a market where the same investment can yield 2.000 times more profit?

2. Maybe the ROI is good, but there is just no budget.

3. There are not enough qualified respondents in the market. There are 2000 babies born in Curacao every year. If a survey with 2% margin of error is desired, you need to interview all the new moms, and then some. How much it will cost in money and time to get to ALL of them? We were recently asked to interview 100 regular users of a minor cigarette brand in a certain age group. How difficult is that? How large is the risk that interviewers or respondents will fake interviews or come up with unqualified respondents?

4. There is not enough time. See point 3

5. Often a high level of accuracy is not necessary. For strategy and marketing purposes we are usually more interested in understanding and exploiting the top 3-4 alternatives, than in making an accurate top 20-ranking. We are interested in the top 3-4 reasons people buy a product, the top 3-4 media vehicles they use, the 3-4 advertising or product enhancing concepts that most appeal to them, the 3-4 closest competitors. All the other reasons, media vehicles, advertising or new product concepts and competitors become irrelevant. Why only the top 3-4? Because those are the ones we will duly consider in strategy development and/or marketing.

6. Borderline cases always mean that your work as a strategist is not done. If 52% like the product concept and 48% dislike, it is imperative to continue working on a better concept, regardless of the margin of error of the study. The risk of the product flopping in the market is simply much too great. What if further enhancement is not possible and a clear winner is not obvious? Well, if budget and time permit and the client wants to assume the risk of a product flopping, you can continue interviewing (making the sample size larger) until the desired accuracy level is reached. I.e. you can start with 400 and work your way up to a sample size of 1000 or more.

7. Segmentation is not very detailed. If the intention is to analyze and develop strategy for different segments, then you need enough respondents in each segment (subgroup) to properly analyze it (there is a formula for that). But, in small markets it is often not profitable to divide the market in many different segments, each with a different strategy. The most we might segment a small market by is 2 or 3 demographic and/or lifestyle variables. The best approach is to first decide which segments you want to study (because you can and intend to develop a separate strategy for them) and decide on minimum sample size later.

Why then have some studies with a sample size of 400 yielded vastly different results from the reality?

There are many reasons other than sample size, that could lead to the “wrong” answer. Following are some of the reasons listed by Aaker, Kumar and Day in Marketing Research (the quintessential marketing research textbook):

1. Interviewers might have phrased questions wrongly

2. Respondents might have intentionally or unintentionally given inaccurate answers

3. The answers might have been recorded, coded, processed or interpreted wrongly

4. Interviewers might have faked the interview. Unsupervised and without validation (i.e. call backs) that is a big risk. Our company asks the name and telephone number of respondents so we can call them back and check (validate). Either that or interviewers are duly supervised.

5. Certain segments (ages, gender, social economic class) might have been over- or underrepresented. Perhaps they could not be found (as young men often are), they were not willing to participate, or participated but systematically refused to answer certain questions. Personally I always participate in surveys but systematically refuse to answer questions about income, politics, religion, sex, etc. That’s one of the reasons we don’t include them in our surveys: never ask a question you would not want to answer.

Sample sizes in the Caribbean

Sample sizes of around 400 are the norm rather than the exception in the small markets of the Caribbean. The constant discussion about sample size inhibits the further development of the research industry. Researchers are afraid to propose small sample sizes, clients are afraid to accept them since they have been led to believe that small sample sizes provide useless answers. Yet, there is no money to pay for more. So, research is forgone, decisions are made on no data at all, while we all know that some data is better than no data at all.

Further reading

Margin of error, Market Research World, Marketing Research by Aaker, Kumar, Day or google the topic.

The Tipping Point at Work in Curacao Referendum

In the previous post http://smallmarketmarketing.blogspot.com/2009/06/lessons-from-marketing-politics-and.html post I made some observations about the referendum campaigns in Curacao. Here is my most interesting observation still.

According to Malcolm Gladwell, "The Tipping Point is that magic moment when an idea, trend or social behavior crosses a threshold, tips and spreads like wildfire". He suggests that these so-called social epidemics are created by a combination of "connectors", "mavens", "salesmen" "stickiness" and "context".

1. Connectors are those people "who know lots of people";


2. Mavens are people known for their objective knowledge and who pass it along without self interest;


3. Salesmen are those communicators who do not only speak but also convince;


4. Stickiness refers to the memorability of the message;


5. The ideal context is when all of the above comes together at the right time.

Nothing really new for a communicator or marketer. But when was the last time you really thought about these?

How do you think the different referendum campaigns in Curacao did on these points?

Lessons from marketing politics and ideas in Curacao

All politics is for local consumption, so too this post dealing with local politics. If you are somehow related to Curacao, you know that on May 15th we had a referendum, of which the result was 52/48.

Throughout the campaigns I was burning to post my observations, but since marketing is not an exact science, I did not want to publicly take the risk.

In the aftermath, what do I think we in Curacao have learnt about political marketing strategy?

The power of emotions

Not just any emotion, the relevant emotions (of the relevant target market); not just emotional content, but the inference, context and form of the emotion. As is the case in consumer products market, marketers who appeal to the right emotions in politics, also do best. Over the past 20 years, marketers, including those in Curacao, have relentlessly trained consumers to choose a product based on emotions (not reason). That's the only decision-making process that today's consumer really knows. He is no longer able to decide based on reason. Hot temperamented people like Caribbeans certainly cannot.

Know your market

Quantify the target segments, understand what motivates each segment and use these core motivators as the starting point for any strategy. Finding a "common motivator" is ideal. But if that does not exist, applying the motivator from one segment across all segments doesn't work, not even in small Caribbean markets.

Mirroring your competitors' position is not a viable strategy

If your competitor positions itself as "for the youngest people", will you position yourself as "for the oldest"? If she says "my product is the cheapest", will you say that yours is the most expensive? Intuitively that does not make sense. Why not, you wonder? Because everyone in the middle is left unserved, untargeted. Marketers know there is a larger market in the middle than at any extreme.

My most interesting observation is in the next post.

The difference between marketing and sales

Seth Godin as a great post on the difference between marketing and sales.

http://sethgodin.typepad.com/seths_blog/2009/05/the-difference-between-marketing-and-sales.html

Guestology

Guestology; Disney term

Not just knowing your guests in terms of demographics, psychographics, need, wants, experiences and expectations.

But understanding your guests: understanding their motivations and analyzing and interpreting objective data in such a way to yield profit enhancing marketing strategies.

The only way to do so is by engaging customers in conversations, through techniques such as focus groups and in-depth interviews.

Analyzing results of Customer Satisfaction Surveys

Here are some interesting observations from Mark David Jones, President of Small World Alliances, and former Disney man:

Dissatisfied customer damage

You think 5% dissatisfied customers is acceptable? Think again. Consider that a dissatisfied customer broadcasts his dissatisfaction 20 to 30 times. And that word-of-mouth is the most powerful form of advertising.

To get the damage done by dissatisfied customers: Multiply your number of annual customers by the percentage of dissatisfied customers. Multiply that number by 20 or 30 for the number of negative advertising images you generate in a year. And remember that w-o-m weighs more than paid advertising.

Extremely satisfied customer earnings

On the other hand, a customer who is merely satisfied is not an advocate. For someone to be your advocate and generate positive word-of-mouth for you, s/he has to be "extremely satisfied". How many "extremely satisfied" customers do you have each year?

If your advertising budget is restrictive, as is often the case in a small market, you know where to invest your money for the most efficiency.

Mark David Jones gave a workshop in Curacao last Friday during CTO's 5th Tourism Human Resources Conference.