SEO Marketing Research

SEO Marketing Research header image 2

How to make survey design

323 Comments · Measurement instruments

Every research problem is unique in some way, and care must be taken to select the most appropriate set of approaches for the problem at hand.

Nevertheless, although every research problem may seem unique, there are usually enough similarities among such problems to allow decisions to be made in advance, as to the best plan to use to resolve the problem and there are some basic survey designs that can be matched to given problems.

There are three basic ways of obtaining primary data in marketing research: survey, observation, and experiment.

Surveys are the most widely used method of data collection in commercial marketing research.

Compared with observation or other qualitative methods, survey methods allow the collection of significant amounts of data in an economical and efficient manner, and they typically allow for much larger sample sizes. There are five advantages of using survey methods.

? Standardization. Questions are preset and organized in a particular arrangement on a questionnaire, and survey methods ensure that all respondents are asked the same questions and are exposed to the same response options.

Thus, the researcher is assured that every respondent will be confronted with questions that address all the information objectives driving the research project.

? Ease of administration. Sometimes an interviewer is used, and survey modes are easily geared to such administration.

On the other hand, the respondent may fill out the questionnaire unattended (sometimes this approach is referred to as a self-explicated interview).

In either case, the administration aspects are much simpler than, for example, conducting a focus group or interviewing.

The simplest method is a postal survey in which questionnaires are sent to prospective respondents.

? Ability to tap the “unseen.” The four questions of what, why, how, and who help uncover “unseen” data.

For example, a working parent may be asked to explain ho\N important the location of a school was in his or her selection of the child’s school.

A researcher can inquire as to how many different schools the parent seriously considered before deciding on one, and go on to gain an understanding of the person’s financial or work circumstances with a few questions on income, occupation, and family size. Much information is unobservable and requires direct questions.

? Suitability to tabulation and statistical analysis. The marketing researcher ultimately must interpret the patterns or themes sometimes hidden in the raw data collected.

Statistical analysis, both simple and complex, is the preferred means of achieving this goal, and large cross-sectional surveys perfectly complement these procedures.

Qualitative methods, in contrast, prove much more frustrating in this respect because of their necessarily small samples, need for interpretation, and general approach to answering marketing managers’ questions.

Increasingly, questionnaire design software includes the ability to perform simple statistical analyses, such as tabulations of the answers to each question, as well as the ability to create graphs summarizing these tabulations.

? Sensitivity to subgroup differences. Because surveys involve large numbers of respondents, it is relatively easy to “slice” up the sample into demographic groups or other subgroups and then to compare them for market segmentation implications.

The large sample sizes that characterize surveys facilitate subgroup analyses and comparisons of various groups in the sample.

Forms of market research
Market research surveys typically take one of five forms:
? Personal surveys, in which face-to-face interviews are conducted at respondents’ homes or offices.

? Intercept surveys, in which face-to-face interviews are conducted with people who are
? stopped at a public location such as a shopping centre.

? Telephone surveys, in which people are interviewed over the telephone.

? Postal surveys, in which people complete self-administered questionnaires that are sent to them.

? Online surveys, in which people fill out a questionnaire that is sent by e-mail.

Personal surveys
The oldest form of survey research is personal interviews conducted at a respondent’s home or workplace.

This method allows control over who answers the questions and maximum flexibility in the questionnaire design.

However, the fact that interviewers must travel to meet respondents makes personal interviewing costly compard with other methods.

Because of the high cost, personal interviewing is rarely used in consumer research. It is mostly used for consumer research in countries where telephone surveys are not culturally accepted, and for surveys of business executives.

Business surveys are often done with respondents who control large budgets and often use open questions that require probing and elaboration.

The individual value of these respondents and the complexity of the interviews justify the expense of personal interviewing.

The flexibility of personal surveys in terms of questionnaire design is unrivalled among the possible methods of data collection. This is because the interviewer can:

? control the order of the questions;
? probe unclear answers and ask complex questions;
? implement branching instructions, which means asking or not asking certain questions depending on the answers to previous questions;
? show the respondent lists of responses to help with questions that have many categories;
? explore unstructured, “conversational” topics;
? conduct a lengthy interview; in fact, interviews lasting an hour or more are not uncommon.

Such flexibility makes personal surveys a preferred method when their cost is not prohibitive. Cost is the big disadvantage of personal surveys, though sample quality is excellent.

Intercept surveys
Because of the high cost of personal surveys, most consumer surveys that use face-to-face interviewing are done by interviewing at a much lower cost than in-home surveys because travel costs are eliminated.

The most popular location for “intercept” surveys is in a high street or shopping centre. because they provide access to a general population that is appropriate for most research.

In most intercept surveys, interviewers are sent out to recruit anyone who looks as if she or he might qualify for the survey. People are approached and asked to participate in interviews on the spot.

If they agree, the interview is administered and the respondent is thanked and paid (in some cases). If they refuse, the interviewer picks another person.

Because intercept surveys are done face-to-face, they offer the same flexibility as a personal survey.

The difference is that intercept surveys are conducted under greater time pressure. In most cases, the interview should not take more than a few minutes unless respondents are paid, because people won’t accept a prolonged interruption of their activities.

Sample quality is a weak point of such surveys. This method is not used for surveys of business executives, except for occasional studies at trade shows, because executives are not easy to identify in the street.

Even for consumer studies, it can be difficult to find good intercept sites. Most marketing research intercepts are done in shopping centres, but many refuse access even when offered rental payments. This limits the extent to which respondents will represent a general population.

Even with access to a mall, low co-operation rates impair the quality of the sample. This is not surprising since many shoppers have limited time.

Telephone surveys
Telephone surveys are widely used, especially in consumer research. They offer better population coverage than intercept surveys but are limited by the fact that interviewers cannot show things to respondents.

In surveys of organizations, companies are selected and phone numbers are found for those companies.

In consumer surveys, the telephone numbers are drawn directly from a directory or with a technique called random digit dialling.

Once the sample is drawn, interviewers call the selected people (or selected numbers) and seek co-operation with the survey.

If the desired respondent isn’t in, the interviewer asks for a good time to call back and tries later. Call-backs should continue until:

? the interview is completed;
? the interview is refused;
? the potential respondent is found to be ineligible; or
? a limit on the number of call-backs is reached.

Telephone interviewers generally work from central offices and most big research companies use computer-assisted telephone interviewing (CATI) systems.

In a CATI system, the interviewer sits in front of a screen on which the questions appear and enters the answers directly into the computer.

Telephone surveys are administered by interviewers, so they offer control of the question sequence, the ability to implement branching instructions, and the opportunity to probe unclear answers.

Since telephone interviews are not conducted face-to-face, it is not possible to show products, packages, or advertisements to the respondent unless they have been sent in advance.

Also, it is not possible to provide respondents with printed lists of responses for questions with many options.

Because of this, and because respondents must be able to keep the question and the responses in their heads, questions should be simple and the number of responses limited.

Sometimes these shortcomings are circumvented by using the TMT approach (telephone-mail-telephone).

According to this method, a respondent is phoned and asked whether the agency may post an information pack containing a sample product, a questionnaire with some pictures, and so on.

The respondent is asked for a date and time when the agency is allowed to call back for a follow-up interview. The interview can be carried out with the respondent having the necessary items to hand.

Interviews up to twenty minutes long are common and do not cause problems. However, it is possible that a survey call will come at a time that is inconvenient for the respondent.

Since the interviewer cannot see this, it is polite to ask if the respondent is free to talk. If not, an appointment should be made for another time.

The sample quality of telephone surveys depends on:

? whether the potential respondents have telephones;
? whether those phones are identified in lists such as the telephone directory; and
? whether the potential respondents are willing to participate.

The main sampling problems with telephone surveys are non-availability and lack of cooperation.

The people who are contacted on the first call are likely to be housewives or retired people.

This creates a bias toward less active people unless the survey organization calls back until designated respondents are reached. Non-co-operation is difficult to solve.

One way to reduce both refusals and non-contacts is to broaden the definition of who is eligible to respond.

Surveys that accept information from any member of a household usually get higher co-operation rates than surveys directed at specific individuals.

An intermediate approach is to ask for a specific individual. If that person refuses or is unavailable, to ask whether any other member of the household is knowledgeable about the survey topic and interview that person instead, as long as the substitute respondent fits the required profile.

One factor that may significantly reduce co-operation rates is significant consumer dissatisfaction with telemarketing.

Telephone surveys of business populations present different issues from those posed by consumer surveys.

The problem in industrial surveys, apart from obtaining cooperation, is locating the right person within the organization to interview.

Usually, this is done by starting with a job title and then getting referred from one person to another until the right person is reached.

The telephone is ideal for this. In fact, industrial postal surveys generally require advance telephone calls to identify names, titles, and exac: addresses of people to whom questionnaires should be sent.

Once the proper respondents are identified, reaching them may still be a problem. A secretary or other “gatekeeper” may limit access to them.

Postal surveys
In a postal survey, a sample of addresses and names are drawn from a list, and the quetionnaire is sent out with a cover letter.

Some studies begin with an advance postcard to explain the survey and alert potential respondents that it is coming.

Approximately two weeks after the first questonnaire, a follow-up questionnaire and letter are sent to non-respondents.

Two weeks later, a second follow-up is sent. If total response is not satisfactory after this second follow-up, it is possible to use telephone interviews to survey a sample of the non-respondents and measure whether they are different in some way from respondents.

A limitation of postal surveys is that they can be used only for short surveys with mainly closed questions. Response rates drop sharply if the questionnaire is longer than four pages.

The questionnaire should offer response categories or at least not require substantial amounts of writing.

This reluctance to answer open questions stems not only from the time and effort required but also from the fact that many respondents are uncertain of their spelling and grammar and do not want to be embarrassed.

Open questions in such surveys usually reduce the co-operation rate substantially while yielding little information.

Another complication is that question order cannot be controlled, as it is in personal interviews.

You have to assume that respondents will read all the questions before answering any of them.

This makes it possible for questions at the end of the questionnaire to influence questions at the beginning, which may sometimes be undesirable.

Questions intended to measure respondents’ level of knowledge about a product, service. or issue don’t work well because respondents are free to look up the answers or ask someone else.

Low response rates create a risk of high non-response bias (i.e. risk of large differences between data for the overall population of interest and data for those who responded).

There are two main sources of non-response bias in postal surveys with low cooperation rates.

First, because surveys of the general population require respondents to have reasonable reading skills, co-operation rates are generally higher for people with higher levels of education.

A second, and more serious, issue is that co-operation on such surveys is influenced by respondents’ interest in the topic.

In attitude surveys, those who feel strongly about something are more likely to respond than are those who don’t care.

For new products or services, those who are interested are more likely to respond, producing overestimates of market interest.

These biases become smaller as sample co-operation increases, but they never vanish from postal surveys.

Unlike telephone and face-to-face surveys, in which people refuse before they really know anything about the topic, you must assume that non-respondents to a postal survey looked at the questionnaire and decided they weren’t interested.

Another sampling problem is that you can’t be certain who the respondent is, and some estimates suggest that one-third of consumer postal surveys are filled out by someone other than the designated respondent (Loosveldt et al., 2004).

In both industrial and consumer research some companies offer postal panels for survey purposes. These panels are households that have agreed, often for incentives, to answer surveys.

These panels are usually balanced by education and other demographic variables to reduce sample bias.

They offer a good way to get relatively quick, relatively high response rates at low cost. However, they cannot be viewed as random samples.

Response rates in male panels may approach 80 percent if the topic is interesting to respondents (political issues) or if good incentives are provided.

But typical panels of, 1,000 or 2,000 respondents cause other methodological problems to the analyst.

From a technical perspective, measurements obtained over time from a panel are times series or multiple measures of the same units and are therefore not independent measures.

Since statistical formulas and experimental designs often assume independence, a panel violates these assumptions.

For instance, a panel is inappropriate for measuring brand awareness before and after a campaign.

Why? Assume that we ask members of the panel whether they know a brand before the campaign, and ask them the same question again after the campaign.

If their knowledge has increased, what has caused the increased awareness? The campaign or the fact that panellists remember the brand because it appeared in the questionnaire received before the campaign?

Remedies for adjusting data, computations and formulas when panel data are involved are discussed in Finkel (1995) and Hsiao (2002).

Online surveys
The popularity of e-mail/internet surveys has surged. There are several reasons for this:
? The speed with which a questionnaire can be created, distributed to respondents, and the data returned.

Since printing, mailing, and data keying delays are eliminated, data can be available within hours of writing a questionnaire.

Responses are in electronic form, so statistical analysis software can be programmed to process standard questionnaires and return statistical summaries and charts automatically.

? Low cost. Printing, mailing, keying, and interviewer costs are eliminated, and the incremental costs of each respondent are typically low, so studies with large numbers of respondents can be done at substantial savings compared with postal or telephone surveys (Ilieva et al., 2002).

Roster et al. (2004) found that the cost for a web survey was 53 percent lower than for a similar telephone survey.

The web garnered a lower response rate, but the findings lend support to the notion that web surveys may be equally, if not more, accurate than telephone surveys in predicting behaviours.

This result is also confirmed by Coderre et al. (2004), whose results show that the quality of qualitative data obtained through a web survey was comparable with that of information obtained through telephone and postal surveys.

? With the creation of respondent panels on the internet, the researcher can create longitudinal studies by tracking attitudes, behaviour, and perceptions over time.

? Sophisticated panel tracking software can tailor follow-up questions in the next survey. based upon responses from a previous survey. Also, missing answers can be filled in.

? Typically, it isn’t worthwhile to conduct a phone survey to ask two or three questions But on the web, a survey component can be included within a general site that is used for marketing or business transactions.

? The ability to reach large numbers of people. The internet is an international arena where many barriers to communication have been erased (Wilson and Laskey, 2003).

? Questionnaires delivered on the web have some unique advantages. They can be made visually pleasing with attractive fonts and graphics.

The graphical and hypertext features of the web can be used to present products for reaction or to explain service offerings. This multimedia ability of web-based questionnaires is unique.

Despite the advantages of online surveys there are still drawbacks. Perhaps the largest problem is that internet users are not representative of the population as a whole.

? Users tend to be male, well educated, technically oriented, relatively young, and have above-average incomes (Schillewaert and Meulemeester, 2005). This is changing. however, as more people access the internet.

? Security. Users are worried about privacy issues. However, given the commercial incentives for insuring that information such as credit card numbers can be transmitted safely, encryption methodology will be at the forefront of internet developments.

? When an unrestricted sample is set up on the internet, anyone who desires can complete the questionnaire.

It is fully self-selecting and probably representative of nothing except web surfers. The problem gets worse if the same person can access the questionnaire over and over.

Recruited internet samples are used for targeted populations in surveys that require more control over the sample.

Respondents are recruited by telephone, post, e-mail, or in person. After qualification, they are sent the questionnaire by e-mail, or are directed to a website that contains a link to the questionnaire.

At websites, passwords are used to restrict access to the questionnaire. Since the make-up of the sample is known, completions can be monitored and follow-up messages can be sent to those who do not complete the questionnaire to improve the participation rate.

Screened-sample questionnaires typically use a branching or skip pattern for asking screening questions to determine whether or not the full questionnaire should be presented to a respondent.

Some web survey systems can make immediate market segment calculations that assign a respondent to a particular segment based on screening questions, and then select the appropriated questionnaire to match the respondent’s segment.

A number of factors or considerations may affect the choice of a survey method in a given situation.

The researcher should choose the survey method that will provide data of the desired types, quality, and quantity at the lowest cost.

Various survey methods each have certain inherent strengths and weaknesses with regard to producing quality data (see the table below).

In some cases, online surveys may be the only way for consumers to respond. In researching the mobility-disabled market, Ray and Tabor (2003) found that these respondents could type and click more easily than write on paper.

While many computer users with disabilities have an internet connection, mainstream advertising to the disabled community has not been well-explored.

International market surveys
The survey methods should be adapted to the specific cultural environment and should not be biased in terms of any one culture.

This requires careful attention at each step of the questionnaire design process. The information needed should be clearly specified.

It is important to take into account any differences in underlying consumer behaviour, decision-making processes, psychographics, lifestyles and demographic variables (Frevert, 2000).

Also, the comparability of data from different countries and cultural contexts has to be assessed.

Comparability in this sense is defined as data that have the same meaning or interpretation and the same level of accuracy, precision of measurement, or reliability.

If data and research design are not comparable from country to country, mistaken inferences may be made about differences or similarities between countries.

In the context of demographic characteristics, information on marital status, education, household size, occupation, income and dwelling unit may have to be specified differently because these variables may not be comparable across countries.

For example, household definition and size varies greatly, given the extended family structure in some countries and the practice of two or more families living under the same roof.

Although personal interviewing may dominate as a survey method in many western countries, different survey methods may be favoured in other countries.

Hence, the questionnaire may have to be suitable for administration by more than one method.

For ease of comprehension and translation, it is better to have two or more simple questions rather than a single complex question.

In overcoming the inability to answer, the variability in the extent to which respondents in different cultures are informed about the subject matter of the survey should be taken into account.

Respondents in some parts of the world may not be as well informed on many issues as people in other parts.

The use of unstructured or open-ended questions may be desirable if the researcher lacks knowledge about the determinants of response in other countries.

What kind of survey method would you recommend to research the question of why female shoppers choose a particular shop at which to buy clothing?

What problems might be encountered by a domestic research company in conducting an international research study?

What problems are faced by researchers conducting research in developing countries?

Keywords: Survey, primary data, data collection, market research, Personal surveys, Intercept surveys, Telephone surveys, Postal surveys, sampling, industrial research, consumer research, postal panels, Online surveys, e-mail survey, internet survey, web survey, International market surveys,


323 Comments so far ↓

Leave a Comment