9 issues to consider when deciding between Telephone or Door-to-door surveys in small markets?

The source for this post is the textbook Marketing Research by Aaker, Kumar and Day, supplemented by our own experience in the small markets of the Caribbean.

When trying to decide between a telephone or door-to-door (face-to-face) survey, these are some issues to consider:

1. turn-around time - Because it is faster to call someone than to drive or walk to them, the turn around time for a telephone survey is shorter than for a face-to-face survey. In our experience, 400 telephone interviews can easily be done in less than 80 man-hours. Not so with face-to-face interviews
2. length of the questionnaire - If the questionnaire is long or complex, it is often not desirable to conduct the interview over the phone. We have found that sometimes respondents are more willing to reveal certain personal information over the phone than in person and vice versa.
3. distance to travel to respondents. If respondents live far apart, a telephone survey might be more cost-effective because of the cost and time of travel.
4. level of supervision. It is easier (and more cost effective) to duly supervise interviewers from a centralized call area, than on the road. Mistakes are also easier caught and corrected, especially if data input happens at the same time, as we do at MarkStra.
5. accessibility to respondents. Not all people have a telephone; interviewers can often not gain access to gated residential communities or apartments; interviewer safety may preclude visits to certain neighborhoods or during certain times of the day. Other groups are systematically inaccessible because of lifestyle or unwillingness to cooperate. Young men are often not home. When they are not, they are likely to be engaging in a group activity and cannot participate, even when called on a mobile phone. Workers in hospitality and health care often work shifts and may not be accessible during "regular" interviewing hours. In Curacao, we find that European Dutch people are systematically unavailable during dinner hours when interviewers are most likely to call or visit.
6. product category. If the survey is about Ringtones, a phone is a pre-requisite. Hence a telephone survey is highly appropriate. If the survey is about a basic necessity (such as water or electricity), some respondents are likely to not have a phone, while their opinion is not only important, but possibly varies significantly from people with phones. If someone has an issue with the electricity company and has a phone, the road to a solution is probably much different than if he doesn't have a phone.
7. sampling method used. Random sampling can be more effectively executed with a telephone survey. It is easier to call and call-back the chosen respondents, than to drive to their house with the risk of not finding them home or the risk of them not qualifying for the survey (f.i. because they are not a user of the product).
8. the number of phones available at the call center
9. budget- total cost is determined among others by the issues above

I started writing this post to explain the limitations of telephone surveys, but have realized that face-to-face interviews have at least as many limitations. Point 5, not being able to enter gated communities and apartments and interviewer safety issues are especially telling.

In the Caribbean, ROI and budget issues are especially important.Telephone surveys probably fit the budget better. Don't dismiss them as being less-than-ideal. Door-to-door surveys have at least as many limitations.

Survey sample sizes in small markets

Earlier this week I had a FB discussion with a fellow researcher about appropriate sample sizes (steekproefgroottes, muestras) in market research. It prompted me to write a post I always want to write each time that discussion comes up in Curacao.

As I write, I realize that the discussion might arise from the different goals of business research versus scientific research. In business (and other organizations) information is gathered to assist in decision-making. It has to make business sense. Scientific research often merely seeks to describe a condition. The findings are not necessarily used to aid in decision-making.

The research field allows for all sample sizes. Researcher just have to indicate the accuracy of the findings.

The larger the sample size, the more accurate the result, i.e. the more likely it is that the result of the study is closer to the reality. What is the error?

Sample (rounded)	Error (rounded)
2.500	2%
1.000	3%
600	4%
400	5%
100	10%

This means that if you interview 400 people and 60% says they like your product, the reality is that between 55% and 65% like your product. If you interview 2.500 people, and 60% says they like your product, the real answer is likely to lie between 58% and 62%.

So, why would a market research not choose to work with the most accurate (large) sample size?

1. ROI reasons. Let’s say that a mass market research study with a sample size of 2.500 costs the same in Curacao, Holland and the USA, e.g. USD 10.000. All else equal, if the accurate answer on which you will base a decision yields you USD 1million extra profit in Holland, it will yield you USD 10.000 extra profit in Curacao – a BREAKEVEN -, just because our market is 100 times smaller (16 million vs 150.000). Why bother? What are the chances that you will sell this research proposal to your US boss, who is accustomed to a market where the same investment can yield 2.000 times more profit?

2. Maybe the ROI is good, but there is just no budget.

3. There are not enough qualified respondents in the market. There are 2000 babies born in Curacao every year. If a survey with 2% margin of error is desired, you need to interview all the new moms, and then some. How much it will cost in money and time to get to ALL of them? We were recently asked to interview 100 regular users of a minor cigarette brand in a certain age group. How difficult is that? How large is the risk that interviewers or respondents will fake interviews or come up with unqualified respondents?

4. There is not enough time. See point 3

5. Often a high level of accuracy is not necessary. For strategy and marketing purposes we are usually more interested in understanding and exploiting the top 3-4 alternatives, than in making an accurate top 20-ranking. We are interested in the top 3-4 reasons people buy a product, the top 3-4 media vehicles they use, the 3-4 advertising or product enhancing concepts that most appeal to them, the 3-4 closest competitors. All the other reasons, media vehicles, advertising or new product concepts and competitors become irrelevant. Why only the top 3-4? Because those are the ones we will duly consider in strategy development and/or marketing.

6. Borderline cases always mean that your work as a strategist is not done. If 52% like the product concept and 48% dislike, it is imperative to continue working on a better concept, regardless of the margin of error of the study. The risk of the product flopping in the market is simply much too great. What if further enhancement is not possible and a clear winner is not obvious? Well, if budget and time permit and the client wants to assume the risk of a product flopping, you can continue interviewing (making the sample size larger) until the desired accuracy level is reached. I.e. you can start with 400 and work your way up to a sample size of 1000 or more.

7. Segmentation is not very detailed. If the intention is to analyze and develop strategy for different segments, then you need enough respondents in each segment (subgroup) to properly analyze it (there is a formula for that). But, in small markets it is often not profitable to divide the market in many different segments, each with a different strategy. The most we might segment a small market by is 2 or 3 demographic and/or lifestyle variables. The best approach is to first decide which segments you want to study (because you can and intend to develop a separate strategy for them) and decide on minimum sample size later.

Why then have some studies with a sample size of 400 yielded vastly different results from the reality?

There are many reasons other than sample size, that could lead to the “wrong” answer. Following are some of the reasons listed by Aaker, Kumar and Day in Marketing Research (the quintessential marketing research textbook):

1. Interviewers might have phrased questions wrongly

2. Respondents might have intentionally or unintentionally given inaccurate answers

3. The answers might have been recorded, coded, processed or interpreted wrongly

4. Interviewers might have faked the interview. Unsupervised and without validation (i.e. call backs) that is a big risk. Our company asks the name and telephone number of respondents so we can call them back and check (validate). Either that or interviewers are duly supervised.

5. Certain segments (ages, gender, social economic class) might have been over- or underrepresented. Perhaps they could not be found (as young men often are), they were not willing to participate, or participated but systematically refused to answer certain questions. Personally I always participate in surveys but systematically refuse to answer questions about income, politics, religion, sex, etc. That’s one of the reasons we don’t include them in our surveys: never ask a question you would not want to answer.

Sample sizes in the Caribbean

Sample sizes of around 400 are the norm rather than the exception in the small markets of the Caribbean. The constant discussion about sample size inhibits the further development of the research industry. Researchers are afraid to propose small sample sizes, clients are afraid to accept them since they have been led to believe that small sample sizes provide useless answers. Yet, there is no money to pay for more. So, research is forgone, decisions are made on no data at all, while we all know that some data is better than no data at all.

Further reading

Margin of error, Market Research World, Marketing Research by Aaker, Kumar, Day or google the topic.

Analyzing results of Customer Satisfaction Surveys

Here are some interesting observations from Mark David Jones, President of Small World Alliances, and former Disney man:

Dissatisfied customer damage

You think 5% dissatisfied customers is acceptable? Think again. Consider that a dissatisfied customer broadcasts his dissatisfaction 20 to 30 times. And that word-of-mouth is the most powerful form of advertising.

To get the damage done by dissatisfied customers: Multiply your number of annual customers by the percentage of dissatisfied customers. Multiply that number by 20 or 30 for the number of negative advertising images you generate in a year. And remember that w-o-m weighs more than paid advertising.

Extremely satisfied customer earnings

On the other hand, a customer who is merely satisfied is not an advocate. For someone to be your advocate and generate positive word-of-mouth for you, s/he has to be "extremely satisfied". How many "extremely satisfied" customers do you have each year?

If your advertising budget is restrictive, as is often the case in a small market, you know where to invest your money for the most efficiency.

Mark David Jones gave a workshop in Curacao last Friday during CTO's 5th Tourism Human Resources Conference.

Take the longer survey “Participating in the global economy through knowledge sourcing and providing”

If you are really interested in the topic and can spare a few more minutes, please also take the survey. If you can, please also ask a colleague, client or supplier to do so.

Knowledge Service Provider http://1236735131.provideknowledge.sgizmo.com/

Knowledge Service Sourcer http://1236735131.sourceknowledge.sgizmo.com/

Thank you in advance. If you want a copy of the report, be sure to include your email address.
Please feel free to leave any comments. The survey is in its beta phase, so I can use comments on the questionnaire also.


Some background

Reason for the poll and survey

Globality: everyone wanting to sell everything, including their knowledge, to anyone anywhere in the world.

Having worked across the Caribbean from Bermuda to Haiti and much of what is in between, I have developed an academic interest in the topic, especially with regard to small states. Can we apply strategic/business innovations produced in Curacao to St Lucian, Maltese, Andorran, Bajan, Cape Verdian businesses, markets and economies and vice versa?

The topic also interests me with regard to economic policies of small states. How can small states benefit from the globalization of knowledge?

Research Question
To what extent do KSP’s sell their services across borders, thus participating in the global knowledge economy? This with particular interest to KSP’s in small ststes/territories/markets and/or emerging markets

What are Knowledge Service Providers (KSP's) and Knowledge Service Sourcers (KSS)?

For the purpose of this study KSP’s are individuals or companies such as advisors, consultants, designers, health care providers, trainers, researchers, recruiters, lawyers, accountants, even artists, etc., in the broadest sense of the word.

KSS’s are all those individuals or companies who buy services from KSP’s established in their market or elsewhere

What's in it for you?
Since participants can elect to receive a copy of the report, you will get an insight into:
1. If you are ahead or behind the curve in terms of providing or sourcing knowledge across borders
2. What the key success factors to sell your services internationally

Take Poll: do you participate in the global knowledge economy?

I am writing a paper and want to know to which extent knowledge service providers (KSP), i.e. consultants, recruiters, trainers, designers, analysts, etc., have international engagements, and thus participate in the global knowledge economy. I am particularly interested in small KSP’s in small and/or emerging markets.

If you are a provider, please follow this link to my Linkedin poll http://polls.linkedin.com/p/35300/cpurr.

If you hire consultants, use this link http://polls.linkedin.com/poll-results/35302/cmmyp.

If you are really interested in the topic, please also take the survey
Knowledge Service Provider http://1236735131.provideknowledge.sgizmo.com/
Knowledge Service Sourcer http://1236735131.sourceknowledge.sgizmo.com

For more info
http://smallmarketmarketing.blogspot.com/2009/04/take-longer-survey-participating-in.html

Thank you, also for your comments

What to expect from a telephone survey

In my previous post I outlined when telephone surveys may be chosen over face-to-face (personal) surveys. What should you expect when you are called for a telephone survey?

1. An interviewer that clearly states his/her first and last name and the company s/he is calling from.
2. If you are called at an inconvenient time, you may ask to be called at another time. In our case, interviews are conducted from a central location. You may also call back.
3. While the interviewer may ask your name, the information you provide will be treated confidentially. Your name is important for the research company when double checking the interviewers' work. It will not be released to our client, NEVER.
4. A questionnaire that is designed to last no longer than 10 minutes. The actual length often depends on how much commentary the respondent delivers. You should know that, at each question, the interviewer can only record one of the options (multiple choice answers) she gives you. She cannot record any other comment. So, it is best to limit yourself to choosing one of the options. At the end of the interview you will always have a chance to make free flowing comments.
5. You will not be called by a sales representative based on your answers. You will not be asked questions in a manner that suggests that your answers will be used to call you later to sell you something. This is a market research exercise, not an exercise to generate sales leads.

Telephone surveys vs face-to-face surveys

You may wonder why companies elect to do telephone surveys rather than face-to-face-surveys. We do so mostly for efficiency and speed. Convenience for the respondent and quality control may also play a role. How so?

1. When the questionnaire is short (up to 10 minutes), it may take more time to travel from one respondent to another. Telephone interviews are more efficient.


2. When the type of respondent are geographically dispersed, it may also take more time to travel from one respondent to another.


3. The respondent may be very mobile or finds it inconvenient to make an appointment that lasts only 10 minutes. The same matter would be resolved faster over the phone.


4. Since telephone interviews are usually conducted from a central location, under supervision, it is easier to control the quality of the interviews.

Telephone surveys are less personal. When the survey entails sensitive issues, this is always a reason try to decrease the respondent's possible discomfort. to For that reason, it is desirable, whenever possible, to inform respondents that they will be contacted and to give some background of the purpose of the survey. Read also my post What to expect from a telephone interview.

The greening of marketing: the 60-30-10 rule

When I started MarkStra in 1995, the Guerrilla Marketing Handbook by Jay Levinson and Seth Godin provided ingenious, effective, practical and low-cost marketing thoughts and tips for my own company and our clients. Here is one:

"When planning a direct mail campaign, remember to follow the commonly known 60-30-10 rule.

• Sixty percent direct mail success lies in using the right mailing list;


• Thirty percent depends on your making the right offer (that is, an offer that satisfies the customer's needs better than the competitor and provides value to him or her);


• Ten percent depends upon your creative package"

Between 1995 and 2007, the price of oil (and consequently also of our marketing material) has risen from just over US$20 per barrel to US$100 and the environment is under pressure.

Have you stopped to evaluate if its worth it, both for your bottom line and for the environment, to produce some of the super-creative marketing material we marketeers strive for? Especially when you know that super-creative material is not exceedingly effective in attracting customers?

Branding Curacao's International Financial Sector

Some 18 months ago, out of pure interest, I started examining what could be the unique competitive advantages of Curacao's international financial sector. The many years of expertise suggest that a unique competitive advantage must exist. Only, I believe the sector has been promoting benefits that now exist in most jurisdictions.

Recently a picture of a beautiful Curacao monument used in a some promotional material for the sector, triggered a discussion and further thought. First about the image the sector really wants to portray. Second, about what potential clients are impressed by and therefore what image (brand) the sector should portray. And then we discussed what picture would best fit with what these clients look for in a Caribbean jurisdiction and therefore what picture we should use. A classic brand essence exercise, but done informally.

I do not know. As many Curacaoans I am proud of our monuments and the fact that our city is on the UNESCO World Heritage List.

Monuments are classic and depict a proper conservatism. That image may very well be necessary. I also know that in at least some sub sectors of the industry, the people are young, risk-takers, educated at some of the premier schools of the world. Now everyone is connected 24/7, the world is fast, global. Unfortunately, in international business, we expect images to be the same wherever we are. That suggests a "seamless fit".

So, it merits further thought and discussion. Most people in the industry are accountants or lawyers, who are less inclined to truly "market". That is not their strength. Proper market research, followed by the appropriate strategy would be a good step to take.

7 ways to use market research

Often we think of market research as a way to get insight from customers with regard to our own company. But, that's not its only use. Here are some ideas:

1. measure, rank and score yourself on issues that are important to your organization (Key Performance Indicators) or the customer (Customer Core Values). Don't bother measuring things that are not important to either you or the customer.
2. check out your competitors. So, you know who to copy (and in which way), whose customers to go after (because they are dissatisfied) or leave alone (because they are way too loyal). Maybe you are uncomfortable asking about your competitors, but an independent researcher is not. That's their job!
3. find out if the perceptions that employees have of their organizations matches the perception of customers and make adjustments when and where necessary.
4. establish a common starting point (a common data-backed assumption) in your organization with regard to opportunities. This, so that innovative ideas are not shot down or delayed just because of different suppositions. This is especially true when the decision-makers have varying degrees of experience with the issue or look at it from different angles. An example: the director and supervisor may have different assumptions. Whose is most likely to be correct? Whose is most likely to "win" without independent insight.
5. determine desired ROI. For example: Based on the research data, can you gain 5% market share? What would be the source of this growth? How much would it cost? Is it worth the effort?
6. If your organization is not yet into ROI, you can use the data to set quantifiable and realistic objectives. Example: if the research shows that you have a 50% market share, is it realistic to set an objective of 25% growth over the next 12 months? In other words, you need to know what your market share is. Otherwise you can't determine if your growth objective is realistic.
7. create a unique value-added experience. Nowadays it's not just the product that is important. It is the experience when purchasing or perusing the product, online or offline, that is. Through research (mystery shopping) you can find out what customers find a "wow" experience.

Which market research method when? A quick guide

Often, when people think about market research, they tend to think about surveys (in Dutch: enquetes; in Papiamentu: enkuesta), with a few hundred interviews.
But, in reality there are several methods of research. In this post I will give a quick run-down of three methods and a quick guide on which to choose when.

In a small market when the incremental benefit/profit from the research findings for your organization may be limited, it is important to make the right choice. It is also important to know in advance how you are going to use the data and to make sure you have the resources to use the findings.

3 Basic Market Research methods

1. quantitative research - mostly to measure in a numerical way. The result will be a score, a ranking. It will reflect issues such as frequency, (average) amount of usage, market share, preference.
2. qualitative research - mostly to explore the breadth of opportunities, innovative ideas, possible service improvements, possible next trends. This can be done through questioning or observation (looking at how exactly customers use a product).
3. desk (secondary research) - using existing data, public or private, local or international to extrapolate, draw inferences and make forecasts.

When to use quantitative research?

1. when you already have some relevant specific information. For instance, if you already know what guests find important in a hotel (the core values), you can then measure how your hotel scores with regard to those core values.
2. when you plan to do the same research regularly in the future. For instance, if you are going to invest in customer service training, you can measure over time if and how much this has improved service. Or, if you plan to increase your rates, you can measure, over time, how this affects your occupancy and guest satisfaction and perhaps that of your competitors.
3. when producing a number, a score or a rank is important because "the boss" wants to see a number or the organizational culture expects to see a number. That's a quite valid reason.

When to use qualitative research?

1. When you have no information at all. For instance, you have no idea how and based on what people choose a hotel, restaurant or car rental. Sometimes the reasons customers act the way they do are by no means obvious or rational.
2. When you want some customer insight, feedback, suggestions, ideas with regard to innovative product or service improvements, branding, pricing.
3. When you really want to understand the customer's deeper thought processes. Qualitative research is most often an interactive conversation, sometimes in a group. Ideas, reasons, perceptions and preconceptions can be challenged, probed, explained further, enhanced, struck down. As markets become more saturated, it is the deep understanding of the customer's motives that provide ideas as to how to target the customer better. Knowing this well and being the first to know, provides a competitive advantage.

When to use desk research?

1. when you are in the orientation or initial phase of a new product or service idea. Often you have no information at all. And, since you are not sure whether the idea will fly, you may not want to make a large investment.
2. When you know that a lot of data is already available, locally or internationally. You can always draw preliminary conclusions based on data that is not your own. The Internet is a tremendous help for you (and both a colleage and a competitor for us). An often overlooked source are market research agencies. They have insight into many industries based on previous work. Yes, MarkStra does too, especially with regard to (alcoholic) beverages, health care and pharmaceutics, banking and Caribbean markets in general.
3. When, quite simply, you don't have the budget to collect your own primary data.

It's official: MarkStra strengthens alliance with Research & Research

We are happy to announce that our firm, marketing and strategy consulting firm, MarkStra, has strengthened our alliance with research specialists Research & Research of Puerto Rico, in order the provide more efficient quantitative market research to our clients.

We have provided market research services since our inception in 1995. Qualitative research (focus groups) and desk research have been our strong suits. In fact, I have been a Burke Certified Focus Group Moderator since 1998. But, our small market combined with rapidly developing technology and increasing customer expectations have made providing quantitative research at international standards a challenge.

Now we can meet that challenge. This alliance will allow us to tap into the expertise and technology of Research & Research whenever needed. We liaise with the local client. Being in Puerto Rico, Research & Research is used to doing research in the Caribbean and Latin America and to working with global brands because several Caribbean headquarters for global brands are established there. That's an added benefit.

I truly look forward to this new form of cooperation. We have worked with Research & Research and several other marketing research firms across the Caribbean for more than 10 years now. What is new is integrating our technology and knowledge to serve our clients and further our businesses. That's the strategic innovation.

How do you use the Internet?

Last semester I taught a marketing course at Curises. That was one of the issues that encouraged me to start exploring the possibilities of the Internet again. The last time I was really involved with Internet opportunities was in 2000, just before the bust.
What amazes me is how the Internet has evolved, how users and providers have found so many ways to use the technology. I counted 24 "basic" things.
What I would really like to know is what my colleagues, friends and business relations, use the internet for and how much.
Please visit http://websurveyor.net/wsb.dll/87513/InternetUsage.htm and fill out the short questionnaire before February 15th. I will post the results as soon as available.
If you have usage alternatives I have missed, please let me know.