SERVICE/PORTAL NAME: NPA/Plus, File 621 in Dialog Classic on the Web (http://www.dialogclassic.com), and also available through Dialog’s proprietary software service. Available through some industry channels on Dialog1 on the Web.

Source description: Gale Group New Product Announcements/Plus (NPA/Plus) is more than just new product announcements (thus the “Plus”).

Anyone who has spent time weeding through article database results chock-full of press releases knows how prolific some companies can be with this type of material.

Although an annoyance in some circumstance, press releases can also be valuable when you are tracking the release of new products and a company’s publicly stated strategies and directions. NPA/Plus serves this purpose to a tee.

The NPA/Plus database is available through all of the iterations of Dialog, through the Web, and through the Dialog software.

For the purposes of this review I evaluated the product by using Dialog Classic on the Web (http://www.dialogclassic.com) and the Dialog 1 News and Industry Channels, also on the Web (http://www.dialog1.com).

An understanding of basic Dialog search syntax will be useful to make the most of this product, but the database can be searched with ease, using a few simple commands.

Pricing: $3.45 per full-text press release, plus connection costs. The database is priced through Dialog, based on the amount of material actually retrieved and viewed, so reular users will wlant to familiarize themselves with the overall pricing structure of Dialog.

Source content: The press releases included in the NPA/Plus database go back through 1985 and are updated daily with current content.

Now numbering more than one million separate releases, they come from the PR Newswire, Business Wire, and other press release wires.

The material included covers just about every industry, business, and geographical area, with no particular emphasis.

It is important to note that, unlike coverage of a company or subject in business publications, press release coverage depends entirely on the company itself, and the depth of information available in the NPA/Plus database will be determined by their public relations efforts.

The full text of the press releases as put out on the wires by the issuing company is presented, indexed by Gale with a handful of searchable fields. These include company name, trade or branch name, and product category name.

Note in the use of the question mark (?) for truncating the terms to pick up both singular and plural. Also note the combination of sets and the limiting by publication year (PY=2002:2003) to narrow down the search results to the most recent material.

The data can be exported as full text or just headlines, though advanced users of Dialog may want to try other search result options, such as keywords-in context.

Keep in mind that most companies append a bit of self-hyping material at the end of their press releases, so if you are passing this material on to others, you may want to remove this material from all but the first press release on the same company.

The Dialog search syntax used to get this list, “T 8/6/20-23,” basically says “Type out the results for Search Set 8 in Format 6 (brief citations) for the 20th to 23rd articles in the search results.”

NPA/Plus provides only press releases and not any proprietary material from trade publications and business journals, so the researcher might be tempted to ask why one would pay for what is essentially free content.

Certainly, one alternative would be to track down the Web site of the company or organization you are interested in and then search for press releases on that one site.

Both PR Newsire and Business Wire also provide a limited amount of free current material on their Web sites, with archival material available for a fee.

But none of the free sources of press releases provide the full search functionality of NPA/Plus, or the depth of coverage.

Source evaluation: Keeping in mind that this resource is providing only an easier way to search and retrieve what is essentially free material – company press releases – you would be hard-pressed to find another source as comprehensive and as easily searchable as Gale’s New Product Announcements/Plus.

Given its limited content, it might be easy to dismiss NPA/Plus as unnecessary. However, it remains a critical resource for looking at a competitor or an industry over time or trying to track new product announcements in a given space.

It is also a valuable source for legal work involving identifying “prior art” and other historical material.

Back files of historical press releases are available through other sources for a fee, but with the NPA/Plus file you get a comprehensiveness that other products simply do not provide.

A savvy searcher who does not necessarily need to see the full text of each press release but just wants to see the titles and dates can get a lot out of this product without incurring too much cost.

The data in NPA/Plus is extremely timely, up to the current date, and goes back through 1985.

There are no limitations on older records, so finding a company’s press releases from 1985 is just as easy as finding those fjrom 2003. Again, knowledge of general Dialog search syntax is useful here, especially whenlimiting by date.

The coverage is only as extensive as what is provided by the companies issuing press releases, and can be representative only of what companies actually take the trouble to put out.

Not finding something mentioned in the NPA/Plus database is no indication that it doesn’t exist, nor should a plethora of press releases on a product be taken as an indication of the product’s actual popularity.

In general, it is quite easy to search this resource, using either free text terms or the company name, trade name, and product-type fields.

If I had not been familiar with the Dialog interface from previous research, I would probably have found the product a bit harder to use.

An important tool is the Bluesheet for the product, which can be downloaded from the main Dialog site (http://library.dialog.com/bluesheets/html/b10621.html) and is well worth consulting before beginning any research.

Bluesheet includes all of the searchable fields in this database, a list of charges for the product, and a thorough product description.

It is unlikely that any researcher will have access to only this one file and not to other material in Dialog, so the technical support would include both general Dialog support and support from Gale for this specific product.

Either way, both Dialog and Gale offer plenty of support for their products, and the folks at Dialog are generally more than willing to walk the researcher through any search process to get the necessary results.

Although contract prices for Dialog can differ greatly, I am not aware of any additional support costs, and it is in the company’s interest to train users to use their products effectively.

Overall, I would rate this as an excellent product for exactly the purpose it is meant to serve: to provide a comprehensive database of press releases going back over time and across all industries.

The one complaint I have with the product is that the “Plus” means that the database now includes financial statements, executive changes, and just about any other press release issues by companies.

It is no longer focused on new product announcements, and that can be a bit frustrating at times when trying to home in on that type of material.

But by using good search vocabulary, even a beginner researcher should have no problem pulling out relevant material.

Source value rating: All of the data source reviewers were asked to rate each source on the basis of the following eight categories, using “10” as the highest rating and “1” as the lowest (“80” being a perfect score):

1. Relative cost-to-value:   7
2. Relative timeliness of data:   10
3. Relative comprehensiveness of data:  9
4. Ease of use:     8
5. Search options available:   9
6. Level of support services:   9
7. Level of training offered:   8
8. Amount/kinds of special services offered: 7
Total Rating     67

Useful tips
• Use company, trade name, and product-type codes to obtain the most relevant search results.

• Remove duplicate material in numerous releses from the same company.

• Keep in mind that all material is self-generated and does not necessarily represent any type of third-party verification or objective valiadation of information in the relese.

Report This Post

The questionnaire in quantitative research is used as a survey instrument with larger samples, normally containing structured questions for ease of coding and analysis.

A questionnaire is a data collection instrument, formally setting out the way in which research questions should be asked.

Even simple questions need proper wording and organization to produce accurate information.

Consideration needs to be given to how questions should be worded, in the light of the objectives of the research, and the target group of respondents. Attention also needs to be given to the organization of the questionnaire and to testing.

There are limitations to what a questionnaire can measure, especially when it comes to product recalls because people can only accurately report upon the item they have bought for a limited time after the actual purchase.

A questionnaire has specific objectives:
? To translate the information needed into questions that the respondents can and will answer. Developing questions that will yield the desired information is difficult.

Two apparently similar ways of posing a question may result in different information. Hence, this objective is a challenge.

? A questionnaire must motivate the respondent to become involved in the interview, to co-operate, and to complete the interview.

Before designing a questionnaire, the researcher must evaluate “what is the respondent going to get out of this” and appreciate what respondents go through when approached and questioned.

Not all respondents are the same in what they seek from a questionnaire or interview process.

? A questionnaire should minimize response error. Such errors arise when respondents give inaccurate answers or when their answers are mis-recorded or mis-analyzed.

A questionnaire can be a significant source of response error and minimizing this error is an important objective of questionnaire design.

To develop a further understanding of questionnaire design, the process can be presented as a series of steps.

The process outlined below shows that the ten steps are interrelated and the development of a questionnaire involves much iteration and interconnection between stages.

Step 1: specify the information needed
The research process often begins when the international marketing manager, brand manager, or product development specialist has a need for decision-making information that is not available.

Five classes of information, generally useful for marketing decisions have been identified:

? Facts and knowledge: what are the beliefs, perceptions, and depth of knowledge of the survey respondents about, for example, specific products, services, industries, or organizations?

? Opinions: what are the existing attitudes towards products, etc., including an assessment of the strength with which these attitudes are held?

? Motives: what motivates buyers of various kinds of products or services?

? Past behaviour: what are the patterns of consumption over specified time periods? Insight will be given into factors such as brand loyalty.

Observational techniques, such as home audits, will help in verifying reported consumption behaviour.

? Future behaviour: indications of possible behaviour may be gleaned from sensitive questioning.

Included in this approach, of course, would be an evaluation of the nature of buying behaviour, which may be complex, and making a list of research objectives.

The first step in questionnaire design is to specify the information needed. It is helpful to review the components of the problem and the approach, particularly the research question, hypotheses and characteristics that influence the research design.

To further ensure that the information obtained fully addresses all the components of the problem, the researcher should prepare a set of variables, which would influence the decision-making problem.

The project’s research objectives should then be translated into information goals that are specific enough to guide question formulation, thinking ahead to the types of tables and graphs that can be used in the final report.

Clearly define the target respondents. The characteristics of the respondent group have a great influence on questionnaire design.

The wording and style of questions that may be appropriate for finance directors being surveyed about their IT needs may not be appropriate for retired persons being asked about their holidays.

The more diversified the respondent group, the more difficult it is to design a single questionnaire appropriate for all.

Step 2: determine the survey methods
A telephone interview often requires a rich verbal description of a concept to make certain the respondent understands the idea being discussed.

In contrast, in an interview, the researcher can show the respondent a picture or demonstrate the concept.

If you want respondents to react to physical stimuli, face-to-face interviews are best. Telephone surveys are very limited in this regard, though it may be possible to send materials to respondents in advance of a phone interview.

Step 3: determine question type and content
Two main types of question response formats are used in marketing research. Openended questions are those in which respondents can reply in their own words.

In other words, the researcher does not limit the response choices. The advantage of such responses is that they can provide the researcher with a rich array of information.

The respondent is answering from his own frame of reference. Advantages are described in “real world” terminology rather than marketing jargon.

Often this is helpful in designing promotional themes and campaigns. It enables copywriters to use the consumer’s language. This rich array of information can now be captured even in computer-assisted interviews.

Open-ended questions are not without their problems. Editing and interpretation of responses are problematic.

If too many categories are used, data patterns and response frequencies may be difficult for the researcher to interpret.

If the categories are too broad, the data are too general and meaning may be lost. Even if a proper number of categories is used, editors may have to interpret what the interviewer has recorded and force data into a category.

A related problem of open-ended questions is interviewer bias. Although training stresses the importance of verbatim recording of open-ended questions, it is often not practised in the field. Also, slow writers may miss important comments.

Open-ended questions also may be biased towards the articulate interviewee. A person with elaborate opinions and the ability to express them may have much greater input than a shy, inarticulate, or withdrawn respondent. Yet they could be equally likely prospects for a product.

So, a basic problem with open-ended questions lies in interpreting and processing data. In fact, a two-phase judgement must be made.

First, the researcher must decide on the proper set of categories and then each response must be evaluated as to which category it falls into.

In general, open-ended questions are useful in exploratory research and as opening or closing questions.

They should be chosen with care as their disadvantages can outweigh their advantages in a large survey.

Closed questions require the respondent to make a selection from a list of responses. The primary advantage of a closed question is simply the avoidance of many of the problems of open-ended questions.

Interviewer and coder bias are removed because the interviewer is simply checking a box, circling a category, recording a number, or punching a key.

The simplest form of a closed-ended question is the dichotomous choice. An example is:
Do you think that this year’s inflation will be greater or less than last year?

? Greater than last year
? Less than last year

Note that the respondent is limited to two answers. It is easy to administer and usually evokes rapid response.

Often a neutral or no opinion/don’t know is added to dichotomous questions to take care of those situations.

Dichotomous questions are prone to measurement error. Because alternatives are polarized, possible choices between the poles are omitted. Thus, question wording is critical to obtain accurate responses.

In the above question, response may vary depending upon whether greater than or less than is listed first.

These problems can be overcome using a split ballot technique. Onehalf of the questionnaires are worded with greater than listed first and the other half with less than first. This procedure will aid in the reduction of potential bias.

As well as open-ended and closed questions, there are multiple-choice questions. These overcome many of the disadvantages of open-ended questions because interviewer bias is reduced and the questions are administered quickly.

Also, coding and processing of data is much less costly and time-consuming. In self-administered questionnaires, respondent co-operation is improved if the questions are structured.

In multiple-choice questions, the researcher provides a choice of answers and respondents are asked to select one or more of the options given.

Sometimes the respondent is asked to only select one item. For instance, a question might ask the respondent “What is your preferred brand?

Alternatively, it could ask the respondent to indicate, say, three favourite brands. A third way would be to simply let the person make the choices that are regarded as relevant or that apply. Consider the following example:

Please indicate all the brands of soft drinks that you have consumed in the past week.
Please check all that apply.

1. Coca-Cola       –
2. Pepsi-Cola      –
3. Fanta              –
4. Seven Up       –
-
-
-
14. Dr pepper       –
15. Other
(please spsescify  –

Of concern in designing multiple-choice questions is the number of options and the order of potential responses, known as position bias.

The response options should include all possible choices. The general guideline is to list the important options and to include another labelled “Other (please specify),” as shown above. The responses should be mutually exclusive.

Multiple-choice questions are not without disadvantages. Considerable effort is required to design effective multiple-choice questions and qualitative techniques may be required to determine the appropriate response options.

It is difficult to obtain information on items not listed. Even if an “Other (please specify)” category is included, respondents tend to choose from the list.

In addition, showing respondents the list of answers produces biased responses. There is also the potential for order bias.

The choice between open- and closed-response questions is not necessarily an either/or distinction.

Open-response questions can be used with closed-response questions to seek additional information.

Using an open-response question to follow up a closed question is called a probe. Probes can combine some advantages of both open and closed questions.

They can be used for specific pre-chosen questions or to obtain additional information from only a subset of people who respond to previous questions in a certain way.

A common example of the latter is to ask respondents who choose “none of the above” a follow-up question to expand on their answer.

There are two general purposes for the use of probes in a questionnaire. The first is to pinpoint questions that were diffiult for respondents.

Adequate testing of questions reduces the need for probes. The second purpose is to aid researcher interpretation of respondent answers.

Answers to open-response follow-ups can provide valuable guidance in the analysis of closedresponse questions.

Step 4: establish question format
Every question in a questionnaire should contribute to the information needed or serve some specific purpose.

It is useful to ask some neutral questions at the start to establish involvement and rapport, particularly when the topic of the questionnaire is sensitive or controversial.

Sometimes filter questions are asked to disguise the purpose or sponsorship of the project.

For example, rather than limiting the questions to the brand of interest, questions about competing brands may be included. At times, certain questions may be duplicated to assess reliability or validity.

Once a question is deemed necessary, the researcher must make sure that it is sufficient to get the desired information.

Sometimes several questions are needed to obtain the required information in an unambiguous manner.

A double-barrelled question is really two different questions posed in one question. With two questions posed together, it is difficult for a respondent to answer either directly.

Consider a question asked of patrons at a restaurant “Were you satisfied with the food and service?” How does the respondent answer?

If they say “yes” does that mean they were satisfied with the food? The service? A combination?

The question would be much improved by asking about a single item: one question for food and another question for service.

Step 5: choose question wording
The wording of specific questions always take a significant amount of time. It is a skill developed over time and subject to constant improvement.

Questions are the raw material of questionnaires and vital to the quality of the research. Question wording is the translation of the desired question content and structure into words that respondents can clearly and easily understand.

Deciding on the wording is perhaps the most critical task in developing a questionnaire. If a question is worded poorly, respondents may refuse to answer it or answer incorrectly.

Even small changes in wording can shift respondent answers, but it is difficult to know whether or not a wording change will have such an effect.

Question phrasing depends on such factors as the information being sought, the characteristics of target respondents, and where the survey is administered. Good questionnaire writing requires that researchers follow these guidelines:

? The questions should be easy to understand. Ordinary words should be used in a questionnaire, and they should match the vocabulary and intellectual level of the respondents.

The most common pitfall is to use technical jargon or specialized terms. Special care must be taken to avoid words that have different meanings for different groups.

This can be readily appreciated in cross-cultural studies, where translation problems are profound.

One socio-economic group may refer to the evening meal as dinner, while others call this meal supper and have their dinner at noon.

Most respondents do not understand technical marketing words. Never forget that you are imposing your language upon respondents in the form of a questionnaire.

Your language communicates and puts respondents in a particular frame of mind as they answer the questions you pose.

Unless that language is meaningful to respondents, they will be put in a frame of mind that you may not intend, and be answering different questions from those you set.

? Questions should be focused on a single issue or topic. The researcher must stay focused on the specific issue or topic.

The question “What type of hotel do you usually stay in when on a trip?” is too vague.

A more focused version is “When you are on a family holiday and stay in a hotel at your destination, what type of hotel do you typically choose?”

? The question should be a grammatically simple sentence. A simple sentence is preferred over compound and complex sentences.

The more complex the sentence, the greater the potential for respondent error. To avoid these problems, the researcher should strive to use a simple sentence structure, even if two separate sentences are necessary for the question.

What is an appropriate length of a question? A common rule of thumb is to limit the number of words in any question to under 20.

Under certain circumstances, a question may have to be long in order to avoid ambiguity, but this should be the exception.

A questionnaire filled with long questions is tiring to answer and more difficult to understand.

Brevity will help respondents to comprehend the central question and reduce the distraction of wordiness.

? Avoid leading questions. A leading question is one that suggests the answer or reveals the researcher’s (or interviewer’s) opinion.

This can be done easily by adding “don’t you agree?” or “wouldn’t you say?” to a desired statement.

A loaded question introduces a more subtle bias. A common type of loading of possible responses is through failure to provide a full range of options, for example, by asking, “How do you generally spend your free time-watching television, or what?” Simply adding “I’m sure you agree” or “Don’t you think?” to a statement can bias responses.

Researchers have also found that respondents tend to agree with plausible propositions unless they have a strong opinion or choices are provided.

Even when options are offered, respondents tend to agree rather than disagree with plausible statements. Given this tendency, questions should be worded neutrally.

? Consider the ability of the respondent to answer the question. Asking respondents about a brand or store that they have never encountered creates a problem.

When a question is worded in such a manner that it implies that the respondent should be able to answer it, then often a reply will be forthcoming, but it will be nothing more than a guess.

This creates measurement error, since uninformed opinions are being recorded. A second problem is forgetfulness.

For instance, “What was the name of the last film you saw on TV?” “Who were the stars?” To avoid the problem of a respondent’s inability to remember, time periods should be kept relatively short.

? Consider the willingness of the respondent to answer the question. Reporting of an event is likely to be distorted in a socially desirable direction.

If the event is perceived as embarrassing, sensitive in nature, threatening, or divergent from one’s self-image, it is likely either not to be reported or to be distorted.

Embarrassing topics that deal with things such as borrowing money, personal hygiene, sexual activities, and criminal records must be phrased in a careful manner to minimize measurement error (Wrobel, 2002).

One technique is to ask the question in the third person. For example, “Do you think that most people spend more using their credit cards than they should?”

By asking about “most people” rather than about the respondents themselves, researchers may be able to learn more about the respondents’ attitude to credit and debt.

A third method for soliciting embarrassing information is to state that the behaviour or attitude is not unusual before asking the question.

For instance, “Millions of people suffer from hemorrhoids; do you or any member of your family suffer from this problem?”

This technique is called “using counter-biasing statements,” and makes embarrassing topics less intimidating for respondents to discuss.

Step 6: arrange the sequence and layout of questions
After questions have been formulated, the next step is to order them and develop a layout for the questionnaire.

Questions should be asked in a logical order. All questions that deal with a particular topic should be asked before beginning another topic.

When switching topics, brief transitional phrases should be used to help respondents switch their train of thought.

“Branching” or “skipping,” a procedure in which certain questions are not asked if they do not apply to a respondent, should be designed carefully.

Branching questions direct respondents to different places in the questionnaire based on how they respond to the question at hand.

These questions ensure that all possible contingencies are covered. A simple way to account for all contingencies is to prepare a flowchart of the logical possibilities and then develop branching questions and instructions based on it.

Branching is most easily done in computer-assisted telephone interviews or online surveys, where software can perform the branching. The logical order of a questionnaire could be:

? Use screener questions to identify qualified respondents. Most market research employs some variation of quota sampling.

Only qualified respondents are interviewed, and specific minimum numbers (quotas) of various types of qualified respondents may be desired.

A study on food products generally has quotas of users of specific brands, a magazine study screens for readers, a cosmetic study screens for brand awareness and so forth.

Thus, any demographics obtained provide a basis for comparison against persons who qualify for the study.

A long screener can significantly increase the cost of the study. It means that you are obtaining more information from every contact with a respondent.

Short screeners quickly eliminate unqualified persons and enable the interiewer to move to the next potential respondent.

Yet a long screener can provide important information on the nature of non-users, or persons unaware of the product or service being researched.

? After introductory comments and screens to find a qualified respondent, the initial questions should be simple, interesting, and non-threatening.

Income or age questions might be disastrous. These are often considered threatening. The initial question should be easy to answer without much forethought.

? Ask general questions first. Once the interview proceeds beyond the opening “warmup” questions, the questionnaire should proceed in a logical fashion.

General questions are covered first to get the person thinking about a concept, company, or type of product, and then the questionnaire moves to the specifics.

? Ask questions that require effort in the middle of the questionnaire. Initially, the respondent is only vaguely interested and understanding of the nature of the survey, until the process builds momentum and commitment to the interview.

When the interviewer shifts to questions with scaled-response formats, the respondent must be motivated to understand the response categories and options.

Build interest and commitment early to motivate the respondent to finish the questionnaire.

? Put sensitive, threatening, and demographic questions at the end. Sensitive topics include money, personal hygiene, family life, political and religious beliefs, and involvement in accidents or crimes.

In industrial surveys, sensitive questions may encompass much of what a company does, especially if it reveals strategy and plans.

Placing these questions at the end ensures that most of the questions will be answered before respondents become defensive or break off the interview.

Moreover, rapport has been established between the respondent and the interviewer by this time, increasing the likelihood of an answer.

Conclude the survey by thanking respondents for their time. You may also want to inform respondents how they can obtain the results of the survey.

The format, spacing and positioning of questions can have a significant effect on the results, particularly in self-administered questionnaires.

It is good practice to divide the form into parts. Several parts may be needed for questions pertaining to the basic information.

Each questionnaire should be numbered. This helps the control questionnaires in the field as well in coding and analysis.

Numbering makes it easy to account for the questionnaires and to determine whether any have been lost.

Finally, don’t make the questionnaire too long. Long questionnaires are tiring and overwhelm respondents.

As respondents exceed the time they have mentally set aside to complete the survey, their responses are no longer accurate. Furthermore, long questionnaires tend to have high non-response rates.

Step 7: obtain approval from relevant parties
Copies of the draft questionnaire should be distributed to all parties that have direct authority over the project.

The client is given the opportunity to comment during the client approval stage, in which the client reviews the questionnaire and assesses whether it covers all of the appropriate issues.

If questions are either inappropriate or perhaps can be improved, it is necessary for the client to convey these changes to the researcher.

This may cause changes, but it is important for the client to approve the questionnaire that will be used.

Client approval ensures that the client is aware of the survey’s progress, and the initialled questionnaire ensures that the researcher is protected against any later claims that the questions were incomplete or done incorrectly.

Step 8: test, revise and correct problems
Once approval has been obtained, the questionnaire should be tested to identify and eliminate problems.

The most basic test is to have as many people as possible look at drafts of the questionnaire as a sounding board. The worst problems will be uncovered by these reviews.

Ideally, a pilot test is done by the best interviewers who will ultimately be working on the job and is administered to target respondents for the study.

They are told to look for misinterpretations by respondents, lack of continuity, poor skip patterns, additional choices for pre-coded and closed questions, and general respondent reaction.

Testing could also involve a trial run of the questionnaire using a small sample of respondents, say five to ten, from the target population.

While the sample may be small, it should cover all subgroups of the target respondents. The goal of testing is to check that the questionnaire will capture the information sought by the researcher.

Testing helps refine the instrument and identifies errors that may be apparent only to the target.

Testing is usually done in two stages. The first stage is personal interviews, regardless of the way the questionnaire will later be administered, because researchers need to observe the behaviours of both the interviewers and the respondents.

Respondents’ reactions to the questions are the primary interest. The interviews can be carried out through protocol analysis or debriefing.

Protocol analysis is an interviewing technique in which respondents think aloud while responding to each question.

Debriefing is an interview conducted after respondents have completed the questionnaire. The respondents are then informed that the exercise was a test and are asked to share with the researcher their thoughts about the question, their answers, and any shortcomings of the survey.

The second testing stage involves administering the survey to a small sample in an environment as similar as possible to the one in which the questionnaire will ultimately be administered.

This stage often reveals problems that cannot be detected in personal interviews. In either phase, researchers should attempt to eliminate any problems and revise the questionnaire.

After the questions have been checked, a numerical code is allocated to each type of response to aid data processing.

All possible answers may be listed and coded in advance of the interview and, in surveys of any size, this is done wherever possible.

If responses cannot be allocated to a range of possible answers, coding can take place after the interview. Especially, pre-coding is relevant in quantitative surveys with closed questions.

Step 9: prepare final copy
Even the final copy phase does not allow the researcher to relax. Precise typing instructions, spacing, numbering, and pre-coding must be set up, monitored, and proof-read.

In general, the quality of copying and the paper used is a function of who will see the questionnaire.

Step 10: implementation
Most research interviewing is conducted by a field services department. It is their duty to complete the interviews and send them back to the researcher. In essence, field services are the production line of marketing research.

Supervisor’s instructions inform interviewers of the nature of the study, start and completion dates, quotas, reporting times, equipment and facility requirement, sampling instructions, number of interviewers required, and validation procedures. Detailed instructions are required for any taste test that involves food preparation.

The supervisor’s instructions are vitally important. Without clear instructions, the interview may be conducted in ten different ways in ten different countries.

Call record sheets are used to measure the efficiency of the interviewers. A form normally indicates the number of contacts and the results of the contact.

A supervisor can examine calls per hour, contacts per completed interview, average time per interview, and similar measures to analyze an interviewer’s efficiency.

If, for example, contacts per completed interview are high, the field supervisor should examine the reasons behind it. Perhaps the interviewer is not using a proper approach or the area may be difficult to cover.
What is the purpose of a questionnaire?

How would you determine whether a specific question should be included in a questionnaire?

What are the issues involved in designing multiple-choice questions?

What are the guidelines available for deciding on question wording?

What is a leading question? Give an example.

Once a questionnaire is developed, what other factors need to be considered before giving it to interviewers?

Why is testing a questionnaire important?

Keywords: Questionnaire, semi-structured Questionnaire, unstructured Questionnaire, Openended questions, Closed questions, split ballot technique,

Report This Post

Nevertheless, although every research problem may seem unique, there are usually enough similarities among such problems to allow decisions to be made in advance, as to the best plan to use to resolve the problem and there are some basic survey designs that can be matched to given problems.

There are three basic ways of obtaining primary data in marketing research: survey, observation, and experiment.

Surveys are the most widely used method of data collection in commercial marketing research.

Compared with observation or other qualitative methods, survey methods allow the collection of significant amounts of data in an economical and efficient manner, and they typically allow for much larger sample sizes. There are five advantages of using survey methods.

? Standardization. Questions are preset and organized in a particular arrangement on a questionnaire, and survey methods ensure that all respondents are asked the same questions and are exposed to the same response options.

Thus, the researcher is assured that every respondent will be confronted with questions that address all the information objectives driving the research project.

? Ease of administration. Sometimes an interviewer is used, and survey modes are easily geared to such administration.

On the other hand, the respondent may fill out the questionnaire unattended (sometimes this approach is referred to as a self-explicated interview).

In either case, the administration aspects are much simpler than, for example, conducting a focus group or interviewing.

The simplest method is a postal survey in which questionnaires are sent to prospective respondents.

? Ability to tap the “unseen.” The four questions of what, why, how, and who help uncover “unseen” data.

For example, a working parent may be asked to explain ho\N important the location of a school was in his or her selection of the child’s school.

A researcher can inquire as to how many different schools the parent seriously considered before deciding on one, and go on to gain an understanding of the person’s financial or work circumstances with a few questions on income, occupation, and family size. Much information is unobservable and requires direct questions.

? Suitability to tabulation and statistical analysis. The marketing researcher ultimately must interpret the patterns or themes sometimes hidden in the raw data collected.

Statistical analysis, both simple and complex, is the preferred means of achieving this goal, and large cross-sectional surveys perfectly complement these procedures.

Qualitative methods, in contrast, prove much more frustrating in this respect because of their necessarily small samples, need for interpretation, and general approach to answering marketing managers’ questions.

Increasingly, questionnaire design software includes the ability to perform simple statistical analyses, such as tabulations of the answers to each question, as well as the ability to create graphs summarizing these tabulations.

? Sensitivity to subgroup differences. Because surveys involve large numbers of respondents, it is relatively easy to “slice” up the sample into demographic groups or other subgroups and then to compare them for market segmentation implications.

The large sample sizes that characterize surveys facilitate subgroup analyses and comparisons of various groups in the sample.

Forms of market research
Market research surveys typically take one of five forms:
? Personal surveys, in which face-to-face interviews are conducted at respondents’ homes or offices.

? Intercept surveys, in which face-to-face interviews are conducted with people who are
? stopped at a public location such as a shopping centre.

? Telephone surveys, in which people are interviewed over the telephone.

? Postal surveys, in which people complete self-administered questionnaires that are sent to them.

? Online surveys, in which people fill out a questionnaire that is sent by e-mail.

Personal surveys
The oldest form of survey research is personal interviews conducted at a respondent’s home or workplace.

This method allows control over who answers the questions and maximum flexibility in the questionnaire design.

However, the fact that interviewers must travel to meet respondents makes personal interviewing costly compard with other methods.

Because of the high cost, personal interviewing is rarely used in consumer research. It is mostly used for consumer research in countries where telephone surveys are not culturally accepted, and for surveys of business executives.

Business surveys are often done with respondents who control large budgets and often use open questions that require probing and elaboration.

The individual value of these respondents and the complexity of the interviews justify the expense of personal interviewing.

The flexibility of personal surveys in terms of questionnaire design is unrivalled among the possible methods of data collection. This is because the interviewer can:

? control the order of the questions;
? probe unclear answers and ask complex questions;
? implement branching instructions, which means asking or not asking certain questions depending on the answers to previous questions;
? show the respondent lists of responses to help with questions that have many categories;
? explore unstructured, “conversational” topics;
? conduct a lengthy interview; in fact, interviews lasting an hour or more are not uncommon.

Such flexibility makes personal surveys a preferred method when their cost is not prohibitive. Cost is the big disadvantage of personal surveys, though sample quality is excellent.

Intercept surveys
Because of the high cost of personal surveys, most consumer surveys that use face-to-face interviewing are done by interviewing at a much lower cost than in-home surveys because travel costs are eliminated.

The most popular location for “intercept” surveys is in a high street or shopping centre. because they provide access to a general population that is appropriate for most research.

In most intercept surveys, interviewers are sent out to recruit anyone who looks as if she or he might qualify for the survey. People are approached and asked to participate in interviews on the spot.

If they agree, the interview is administered and the respondent is thanked and paid (in some cases). If they refuse, the interviewer picks another person.

Because intercept surveys are done face-to-face, they offer the same flexibility as a personal survey.

The difference is that intercept surveys are conducted under greater time pressure. In most cases, the interview should not take more than a few minutes unless respondents are paid, because people won’t accept a prolonged interruption of their activities.

Sample quality is a weak point of such surveys. This method is not used for surveys of business executives, except for occasional studies at trade shows, because executives are not easy to identify in the street.

Even for consumer studies, it can be difficult to find good intercept sites. Most marketing research intercepts are done in shopping centres, but many refuse access even when offered rental payments. This limits the extent to which respondents will represent a general population.

Even with access to a mall, low co-operation rates impair the quality of the sample. This is not surprising since many shoppers have limited time.

Telephone surveys
Telephone surveys are widely used, especially in consumer research. They offer better population coverage than intercept surveys but are limited by the fact that interviewers cannot show things to respondents.

In surveys of organizations, companies are selected and phone numbers are found for those companies.

In consumer surveys, the telephone numbers are drawn directly from a directory or with a technique called random digit dialling.

Once the sample is drawn, interviewers call the selected people (or selected numbers) and seek co-operation with the survey.

If the desired respondent isn’t in, the interviewer asks for a good time to call back and tries later. Call-backs should continue until:

? the interview is completed;
? the interview is refused;
? the potential respondent is found to be ineligible; or
? a limit on the number of call-backs is reached.

Telephone interviewers generally work from central offices and most big research companies use computer-assisted telephone interviewing (CATI) systems.

In a CATI system, the interviewer sits in front of a screen on which the questions appear and enters the answers directly into the computer.

Telephone surveys are administered by interviewers, so they offer control of the question sequence, the ability to implement branching instructions, and the opportunity to probe unclear answers.

Since telephone interviews are not conducted face-to-face, it is not possible to show products, packages, or advertisements to the respondent unless they have been sent in advance.

Also, it is not possible to provide respondents with printed lists of responses for questions with many options.

Because of this, and because respondents must be able to keep the question and the responses in their heads, questions should be simple and the number of responses limited.

Sometimes these shortcomings are circumvented by using the TMT approach (telephone-mail-telephone).

According to this method, a respondent is phoned and asked whether the agency may post an information pack containing a sample product, a questionnaire with some pictures, and so on.

The respondent is asked for a date and time when the agency is allowed to call back for a follow-up interview. The interview can be carried out with the respondent having the necessary items to hand.

Interviews up to twenty minutes long are common and do not cause problems. However, it is possible that a survey call will come at a time that is inconvenient for the respondent.

Since the interviewer cannot see this, it is polite to ask if the respondent is free to talk. If not, an appointment should be made for another time.

The sample quality of telephone surveys depends on:

? whether the potential respondents have telephones;
? whether those phones are identified in lists such as the telephone directory; and
? whether the potential respondents are willing to participate.

The main sampling problems with telephone surveys are non-availability and lack of cooperation.

The people who are contacted on the first call are likely to be housewives or retired people.

This creates a bias toward less active people unless the survey organization calls back until designated respondents are reached. Non-co-operation is difficult to solve.

One way to reduce both refusals and non-contacts is to broaden the definition of who is eligible to respond.

Surveys that accept information from any member of a household usually get higher co-operation rates than surveys directed at specific individuals.

An intermediate approach is to ask for a specific individual. If that person refuses or is unavailable, to ask whether any other member of the household is knowledgeable about the survey topic and interview that person instead, as long as the substitute respondent fits the required profile.

One factor that may significantly reduce co-operation rates is significant consumer dissatisfaction with telemarketing.

Telephone surveys of business populations present different issues from those posed by consumer surveys.

The problem in industrial surveys, apart from obtaining cooperation, is locating the right person within the organization to interview.

Usually, this is done by starting with a job title and then getting referred from one person to another until the right person is reached.

The telephone is ideal for this. In fact, industrial postal surveys generally require advance telephone calls to identify names, titles, and exac: addresses of people to whom questionnaires should be sent.

Once the proper respondents are identified, reaching them may still be a problem. A secretary or other “gatekeeper” may limit access to them.

Postal surveys
In a postal survey, a sample of addresses and names are drawn from a list, and the quetionnaire is sent out with a cover letter.

Some studies begin with an advance postcard to explain the survey and alert potential respondents that it is coming.

Approximately two weeks after the first questonnaire, a follow-up questionnaire and letter are sent to non-respondents.

Two weeks later, a second follow-up is sent. If total response is not satisfactory after this second follow-up, it is possible to use telephone interviews to survey a sample of the non-respondents and measure whether they are different in some way from respondents.

A limitation of postal surveys is that they can be used only for short surveys with mainly closed questions. Response rates drop sharply if the questionnaire is longer than four pages.

The questionnaire should offer response categories or at least not require substantial amounts of writing.

This reluctance to answer open questions stems not only from the time and effort required but also from the fact that many respondents are uncertain of their spelling and grammar and do not want to be embarrassed.

Open questions in such surveys usually reduce the co-operation rate substantially while yielding little information.

Another complication is that question order cannot be controlled, as it is in personal interviews.

You have to assume that respondents will read all the questions before answering any of them.

This makes it possible for questions at the end of the questionnaire to influence questions at the beginning, which may sometimes be undesirable.

Questions intended to measure respondents’ level of knowledge about a product, service. or issue don’t work well because respondents are free to look up the answers or ask someone else.

Low response rates create a risk of high non-response bias (i.e. risk of large differences between data for the overall population of interest and data for those who responded).

There are two main sources of non-response bias in postal surveys with low cooperation rates.

First, because surveys of the general population require respondents to have reasonable reading skills, co-operation rates are generally higher for people with higher levels of education.

A second, and more serious, issue is that co-operation on such surveys is influenced by respondents’ interest in the topic.

In attitude surveys, those who feel strongly about something are more likely to respond than are those who don’t care.

For new products or services, those who are interested are more likely to respond, producing overestimates of market interest.

These biases become smaller as sample co-operation increases, but they never vanish from postal surveys.

Unlike telephone and face-to-face surveys, in which people refuse before they really know anything about the topic, you must assume that non-respondents to a postal survey looked at the questionnaire and decided they weren’t interested.

Another sampling problem is that you can’t be certain who the respondent is, and some estimates suggest that one-third of consumer postal surveys are filled out by someone other than the designated respondent (Loosveldt et al., 2004).

In both industrial and consumer research some companies offer postal panels for survey purposes. These panels are households that have agreed, often for incentives, to answer surveys.

These panels are usually balanced by education and other demographic variables to reduce sample bias.

They offer a good way to get relatively quick, relatively high response rates at low cost. However, they cannot be viewed as random samples.

Response rates in male panels may approach 80 percent if the topic is interesting to respondents (political issues) or if good incentives are provided.

But typical panels of, 1,000 or 2,000 respondents cause other methodological problems to the analyst.

From a technical perspective, measurements obtained over time from a panel are times series or multiple measures of the same units and are therefore not independent measures.

Since statistical formulas and experimental designs often assume independence, a panel violates these assumptions.

For instance, a panel is inappropriate for measuring brand awareness before and after a campaign.

Why? Assume that we ask members of the panel whether they know a brand before the campaign, and ask them the same question again after the campaign.

If their knowledge has increased, what has caused the increased awareness? The campaign or the fact that panellists remember the brand because it appeared in the questionnaire received before the campaign?

Remedies for adjusting data, computations and formulas when panel data are involved are discussed in Finkel (1995) and Hsiao (2002).

Online surveys
The popularity of e-mail/internet surveys has surged. There are several reasons for this:
? The speed with which a questionnaire can be created, distributed to respondents, and the data returned.

Since printing, mailing, and data keying delays are eliminated, data can be available within hours of writing a questionnaire.

Responses are in electronic form, so statistical analysis software can be programmed to process standard questionnaires and return statistical summaries and charts automatically.

? Low cost. Printing, mailing, keying, and interviewer costs are eliminated, and the incremental costs of each respondent are typically low, so studies with large numbers of respondents can be done at substantial savings compared with postal or telephone surveys (Ilieva et al., 2002).

Roster et al. (2004) found that the cost for a web survey was 53 percent lower than for a similar telephone survey.

The web garnered a lower response rate, but the findings lend support to the notion that web surveys may be equally, if not more, accurate than telephone surveys in predicting behaviours.

This result is also confirmed by Coderre et al. (2004), whose results show that the quality of qualitative data obtained through a web survey was comparable with that of information obtained through telephone and postal surveys.

? With the creation of respondent panels on the internet, the researcher can create longitudinal studies by tracking attitudes, behaviour, and perceptions over time.

? Sophisticated panel tracking software can tailor follow-up questions in the next survey. based upon responses from a previous survey. Also, missing answers can be filled in.

? Typically, it isn’t worthwhile to conduct a phone survey to ask two or three questions But on the web, a survey component can be included within a general site that is used for marketing or business transactions.

? The ability to reach large numbers of people. The internet is an international arena where many barriers to communication have been erased (Wilson and Laskey, 2003).

? Questionnaires delivered on the web have some unique advantages. They can be made visually pleasing with attractive fonts and graphics.

The graphical and hypertext features of the web can be used to present products for reaction or to explain service offerings. This multimedia ability of web-based questionnaires is unique.

Despite the advantages of online surveys there are still drawbacks. Perhaps the largest problem is that internet users are not representative of the population as a whole.

? Users tend to be male, well educated, technically oriented, relatively young, and have above-average incomes (Schillewaert and Meulemeester, 2005). This is changing. however, as more people access the internet.

? Security. Users are worried about privacy issues. However, given the commercial incentives for insuring that information such as credit card numbers can be transmitted safely, encryption methodology will be at the forefront of internet developments.

? When an unrestricted sample is set up on the internet, anyone who desires can complete the questionnaire.

It is fully self-selecting and probably representative of nothing except web surfers. The problem gets worse if the same person can access the questionnaire over and over.

Recruited internet samples are used for targeted populations in surveys that require more control over the sample.

Respondents are recruited by telephone, post, e-mail, or in person. After qualification, they are sent the questionnaire by e-mail, or are directed to a website that contains a link to the questionnaire.

At websites, passwords are used to restrict access to the questionnaire. Since the make-up of the sample is known, completions can be monitored and follow-up messages can be sent to those who do not complete the questionnaire to improve the participation rate.

Screened-sample questionnaires typically use a branching or skip pattern for asking screening questions to determine whether or not the full questionnaire should be presented to a respondent.

Some web survey systems can make immediate market segment calculations that assign a respondent to a particular segment based on screening questions, and then select the appropriated questionnaire to match the respondent’s segment.

A number of factors or considerations may affect the choice of a survey method in a given situation.

The researcher should choose the survey method that will provide data of the desired types, quality, and quantity at the lowest cost.

Various survey methods each have certain inherent strengths and weaknesses with regard to producing quality data (see the table below).

In some cases, online surveys may be the only way for consumers to respond. In researching the mobility-disabled market, Ray and Tabor (2003) found that these respondents could type and click more easily than write on paper.

While many computer users with disabilities have an internet connection, mainstream advertising to the disabled community has not been well-explored.

International market surveys
The survey methods should be adapted to the specific cultural environment and should not be biased in terms of any one culture.

This requires careful attention at each step of the questionnaire design process. The information needed should be clearly specified.

It is important to take into account any differences in underlying consumer behaviour, decision-making processes, psychographics, lifestyles and demographic variables (Frevert, 2000).

Also, the comparability of data from different countries and cultural contexts has to be assessed.

Comparability in this sense is defined as data that have the same meaning or interpretation and the same level of accuracy, precision of measurement, or reliability.

If data and research design are not comparable from country to country, mistaken inferences may be made about differences or similarities between countries.

In the context of demographic characteristics, information on marital status, education, household size, occupation, income and dwelling unit may have to be specified differently because these variables may not be comparable across countries.

For example, household definition and size varies greatly, given the extended family structure in some countries and the practice of two or more families living under the same roof.

Although personal interviewing may dominate as a survey method in many western countries, different survey methods may be favoured in other countries.

Hence, the questionnaire may have to be suitable for administration by more than one method.

For ease of comprehension and translation, it is better to have two or more simple questions rather than a single complex question.

In overcoming the inability to answer, the variability in the extent to which respondents in different cultures are informed about the subject matter of the survey should be taken into account.

Respondents in some parts of the world may not be as well informed on many issues as people in other parts.

The use of unstructured or open-ended questions may be desirable if the researcher lacks knowledge about the determinants of response in other countries.

What kind of survey method would you recommend to research the question of why female shoppers choose a particular shop at which to buy clothing?

What problems might be encountered by a domestic research company in conducting an international research study?

What problems are faced by researchers conducting research in developing countries?

Keywords: Survey, primary data, data collection, market research, Personal surveys, Intercept surveys, Telephone surveys, Postal surveys, sampling, industrial research, consumer research, postal panels, Online surveys, e-mail survey, internet survey, web survey, International market surveys,

Report This Post

In addition to all the issues related to achieving comparability and equivalence in the instrument, there is the underlying issue of whether the construct exists and can be measured using the same or similar instrument in more than one culture.

Most published research dealing with cross-cultural scales reports the results where a scale that has been developed in one country, typically the US, is applied in other countries.

Few, if any, modifications are made to the original scale, with the exception of dropping items that do not exhibit high levels of reliability.

In taking this approach. researchers are assuming that a construct found in one country is manifested in the same form in another.

Researchers may also adapt the scale by adding items to enhance their ability to identify culture-specific constructs.

Scale development can take place at the levels of an individual or the country. Most of the scaling literature deals with individual level scales or scales that are based on individuals’ data but ascribed to the country.

Scales can also be developed using macro-country data, in which case they would reflect country characteristics.

Scales may also be based on individuals’ responses as members of an organization. In this case inferences would be made about the organization.

In general, verbal scales are more effective among less-educated respondents, but a more appropriate procedure for illiterate respondents would be scales with pictorial stimuli.

Other devices such as smiling faces and a thermometer scale are also used among less-educated respondents.

Moreover, culture can affect the responses and may induce bias. The Likert scale is culture-bound and should be treated as a culture-specific instrument.

Research has been conducted to find out whether there is a pan-cultural scale. Bearing in mind the drawbacks of administering scales to respondents in different countries, the one scale that has consistently provided accurate results is the semantic differential scale.

Because the adjectives on the polar ends of the scale are opposite in meaning, it is easier for the respondent to understand and answer questions in a manner that is useful to the researcher (Kumar, 2000).

Designing scales for international marketing research calls for a great deal of adaptation on the researcher’s part.

It has to be decided whether a single scale can be used in all of the countries or whether it should be customized for each country.

In the US, a five or a seven point scale is used, but people in other countries, such as France, are familiar with a twenty-point scale.

Semantics plays an important role in the accuracy with which a scale measures any given attribute.

Many cultures tend to overstate their feelings, while others are more modest. The word “excellent” may connote very different levels of perfection to Japanese and Scandinavians.

Adjustments for such linguistic differences have to be made. It has been observed that verbal rating scales work the best in the international context.

All respondents are accustomed to talking about their feelings, irrespective of their country or culture.

In international marketing research we will often find some degree of construct bias in the research process.

Construct bias is likely to be present if the construct being studied differs across countries, or if the operationalization does not fit cultural understanding.

Construct bias can, for example, be induced if behaviours are sampled that are not associated with the construct studied.

The use of butter for baking in one country cannot be compared with the use of butter for spreading in another country and, as a consequence, attitudes towards butter will reflect quite different notions about the use of butter (Herk et al., 2005).

Keywords: cross-cultural researchers, scales, pictorial stimuli, thermometer scale,

Report This Post

This is common in research. Since virtually all research efforts are flawed. Marketing researchers must routinely measure the accuracy of their information.

Researchers must determine measurement error, which is the difference between the information sought and the information actually obtained in the measurement process.

Measurement includes true, accurate information plus some degree of error. We can summarize this idea as follows:

Measurement results = True measurement + Measurement error

Two potential sources of error exist in the measurement process: systematic error and random error. Therefore we can extend the equation as follows:

Measurement results = True measurement + (Systematic error + Random error)

Systematic error is caused by a constant bias in the design or implementation of the measurement situation.

This type of error occurs in samples where the findings are either consistently higher or consistently lower than the actual value of the population parameter being measured.

Systematic error is also referred to as non-sampling error, because it encompasses all types of errors except those brought about by random sampling.

In relation to assessing errors in the measurement instruments, reliability and validity are two central concepts.

Reliability
A reliable scale is a prerequisite to sound research. Reliability refers to the ability of a scale to produce a consistent result if repeated measurements are taken.

If a lecturer gives a group of students two different tests to measure their knowledge of marketing research, and the students’ scores from the two measures are very similar, then the measures can be said to be reliable since they replicated each other’s scores.

Reliability is the extent to which scales are free of random error and so produce consistent results.

In general, the less random error detected, the more reliable the data will be. Systematic sources of error do not lessen reliability, because they consistently influence the measurement rather than create inconsistencies to it.

In test-retest reliability, respondents are given identical sets of scale items at two different times, under nearly equivalent conditions.

The time interval between tests is typically two to four weeks. The degree of similarity between the two measurements is determined by computing a correlation coefficient. The higher the correlation coefficient, the greater the reliability.

That test retest approach is subject to time constraints, which creates another potential problem for the researcher.

The greater the time interval between the first and second tests, the less reliable the scale will be. Also, environmental and personal elements may change and alter the results of the second test.

Another issue is that it is often difficult to persuade the original respondents to take a second test. There may also be a carryover effect from the first measure.

This is called the halo effect. Suppose respondents were initially asked to rate the service of a shop.

Their response to a similar question asked two weeks later may be influenced by their initial response.

A third problem with test-retest reliability is that some situations can be measured only once.

Alternative-forms reliability (sometimes called “equivalent-forms reliability”) is the ability of two “equivalent” scales to obtain consistent results.

To carry out this test, researchers administer one scale to respondents and, about two weeks later, administer the second equivalent scale to the same respondents.

In theory, there should be no carryover effect, because the items are different, so scores from the first scale should not affect scores on the second.

A similar number of questions should be used on each scale to measure the topic under investigation.

After the respondents have completed the two scales, researchers compare the measurement instruments item-by-item to determine how similar they are.

The problem with alternative-forms reliability lies in constructing two scales that appear different yet have similar content.

The alternative-forms test is similar to the test-retest method except that the test-retest method uses the same measurement instrument both times, not two different instruments.

In internal-consistency reliability, two or more measurements of the same concept are taken at the same time and then compared to see whether they agree.

Suppose the following four statements using a Likert scale (choices range from “strongly agree” to “strongly disagree”) are used to determine consumers’ attitudes towards My Bank’s cutomer service: ”

I always enjoy visiting My Bank,” “I like the people who work at Nt. Bank,” “My Bank satisfies my banking needs,” “The services I receive at My Bank are excellent.”

The extent to which the four measures correlate across a sample of respondents indicates the reliability of the measures. As the correlation increases, the reliability of the measures increases.

The easiest way to test for internal consistency is to use the split-half technique This method assumes that these items can be divided into two equivalent subsets tha: can be compared.

Several methods have been devised to divide the items randomly into halves and compute a measure of similarity of the total scores of the two halves acro» the sample.

An average split-half measure of similarity, coefficient alpha, can he obtained from a procedure that has the effect of comparing every item with every other item.

Coefficient alpha (or Cronbach’s alpha) is a technique for judging internal consistency of a measurement instrument by averaging all possible ways of splitting test items and examining their degree of correlation (Cronbach, 1951).

The greater the correlation is to a score of 1, the higher the internal consistency (Cronbach, 1990). A score of 0.60 or less indicates that the items measure different characteristics.

How can reliability be improved?
Here are some ways to improve reliability:
? Increase the number of measurements. Instead of using the scores from one test, sum or average the scores on several equivalent forms of the test.

To do this, increase the number of test items, checking to make sure that the new items examine similar concepts.

? Use good experimental controls. To minimize non-systematic or random factors, the testing situation must be conducive to achieving consistent responses.

Therefore, make sure that lighting is comfortable and consistent, measuring devices such as stopwatches work properly, the measurement scale is reliable and consistent, and test administrators know how to avoid creating bias in respondents.

? Be careful to select only items relevant to the topic for measurement. Define the study topic carefully and correctly and then write test items that will accurately reveal information about that topic.

Validity
Just because a measurement scale produces consistent results doesn’t mean it measures the right concept.

Validity is the degree to which a test measures what it is supposed to measure: “Are we measuring what we think we are measuring?”

All too often researchers think they are measuring one thing when they are actually measuring something else.

There are several ways to assess the validity of measurement instruments: content validity, criterion-related validity, construct validity, convergent validity, and discriminant validity.

One way to judge the content validity of a scale is to ask experts on the test topic to assess the scale.

Scales that pass this test are said to have content validity. This test is subjective because the personal experiences and beliefs of the experts inevitably come into play.

Content validity is the most often used validation technique, because it is not timeconsuming and is easy to do.

Criterion-related validity is the ability of a scale to perform as predicted in relation to a specified criterion.

The criterion is the attribute of interest. The predictor is the respondent’s score. Suppose that a post graduate business school tries to determine applicants’ potential by asking them all to take an admissions test.

The criterion is each applicant’s potential to succeed on the course. The predictor is the applicant’s test score.

What is important is how well the predictor determines the applicant’s potential for success in the course.

There are two types of criterion-related validity: concurrent validity and predictive validity.

Concurrent validity evaluates how well the results from one scale correspond with the result from another when the scales measure the same phenomenon at the same time.

Validity is determined by how closely the results correlate with each other. Concurrent validity also assesses how well a set of independent variables can predict the dependent variable in the light of new information.

Assume that respondents have filled in a questionnaire and that a model has been built based on relationships between variables.

Predictive validity is the ability of a scale to predict a future occurrence or phenomenon. What differentiates this form of validity from concurrent validity is the time period when the tests are administered.

If a brand’s market share one year after the launch is 17 percent and the agency’s market research prior to the launch predicted a share of 16 to 19 percent with a 95 percent probability, this is an example of good predictive validity.

Construct validity concerns an abstract, unobservable, hypothesized concept. Constructs can be characteristics such as intelligence, aptitude, strength, love, and creativeness.

In marketing, constructs that researchers often want to measure include service quality, customer satisfaction, and brand loyalty.

Because of their abstract nature, there is no direct way to measure constructs, so researchers measure observable phenomena that theoretically demonstrate the presence of the construct.

Suppose a researcher wants to measure the quality of a company’s service. Theory states that the amount of repeat business, an observable phenomenon, reflects service quality, an unobservable phenomenon.

If this theory is valid, an instrument can be created that measures repeat business and uses the results as a measurement of service quality.

A scale has construct validity if it measures an observable phenomenon that an underlying theory correlates with the construct of interest.

Stated another way, construct validity assesses how well ideas or theories are translated into real measures.

In this case, a scale has construct validity if it can show that repeat business demonstrates service quality.

The validity of the underlying theory, that repeat business demonstrates service quality, is key to the validity of the scale.

If the theory is wrong and there is no association between the two, then the scale is not valid; that is, it won’t measure service quality even if it does measure repeat business well.

When construct validity is not found, it may be due to either a lack of construct validity or a flaw in the theory.

To avoid these problems, researchers try to establish the construct validity of a measure by relating it to many constructs rather than just one. Researchers also try to use proven theories.
 
Construct validity exists if both convergent and discriminate validity are present.

Convergent validity is the ability of a scale to correlate with other scales that purport to measure the same concept, the logic being that two or more measurements of the same concept using different scales should agree if they are valid measures of the concept.

If the results from different scales that claim to measure the same construct are highly correlated. then convergent validity is established.

Relationship between reliability and validity
Ideally, a measurement used by a market researcher should be reliable and valid. Figure 6. 10 shows various types of reliability and validity measures.

Although this figure treats validity and reliability as being independent of each other, there is actually a one-way relationship between them.

A scale must be reliable to be valid; but it does not have to be valid to be reliable. Further, reliability is a necessary but not sufficient condition for validity, because validity also requires other factors to be satisfied (that is, supported from theory and observation).

Validity is not a necessary condition for reliability. For deeper coverage of validity and reliability see Carmines and Zeller (1979).

Generalizability refers to the extent to which it is possible to generalize from the present sample data to the universe.

Usually, a poll that is based on a nationally representative sample of a thousand or more voters is believed to possess generalizability.

In contrast comprehensive quantitative models of consumer behaviour like the LISREL model sometimes are criticized due to low generalizability.

This happens because the model’s parameters cannot be applied to a universe or market environment outside the one (the specific sample) that was used for building the model.

A study finding or a model may possess good validity and be reliable while its findings are not generalizable.

What are the arguments for and against the inclusion of a neutral response position in a symmetric scale?

Can random error be avoided? If so, how? If not, why not?

How is criterion validity assessed?

When assessing a measurement instrument’s construct validity, why is it necessary for the instrument to have a theoretical foundation?

What is the halo effect, and how does a researcher allow for it?

What are the necessary conditions for a study’s generalizability?

Keywords: Reliability, validity, measurement error, systematic error, random error, test-retest reliability, halo effect, Alternative-forms reliability, equivalent-forms reliability, In internal-consistency reliability, split-half technique, Coefficient alpha, Cronbach’s alpha, content validity, criterion-related validity, construct validity, convergent validity, discriminant validity, criterion-related validity, concurrent validity, predictive validity,

Report This Post

 The main sampling techniques can be divided into probability and non-probability methods.

 In probability sampling each element of the population has a chance of being selected. In such cases it is possible to compute sampling variation and project the results to the entire population.

 In the case of non-probability sampling, the chance of selection of a particular population element is known and, strictly speaking, results cannot be projected to the entire population.

Although sampling can be technically rigorous, the need to be so does depend on the particular application.

 There are two general reasons why a sample is more desirable than a census. First, there are practical considerations such as cost and population size that make a sample preferable.

 Taking a census is expensive as consumer populations may number in the millions. Second, typical research firms or the typical researcher cannot analyze the huge amounts of data generated by a census.

 Although statistical software can handle thousands of observations with ease, programs slow down appreciably with tens of thousands, and most are unable to accommodate hundreds of thousands of observations.

 In fact, even before a researcher considers the size of the computer or tabulation equipment to be used, he or she must consider the various data preparation procedures involved in just handling the questionnaires or responses and transferring these into computer files.

The sheer physical volume places limitations on the researcher’s staff and equipment.

 The sampling process make the sampling experience less complex:

 Step 1: determine the target population
The population is the total group to be studied, the target population (universe).

It is the grand total of what is being measured: consumers, stores, households or whatever.

 If the purpose of the study has been well-defined, the population is also well-delineated.

This is crucial if the study is to be significant and practical for the guidance of marketing management.

 “Target” refers to the conditions that separate those who are of interest to a research project from those who are not. For example, common boundary conditions in marketing research could include:

 ? whether a person has bought the product in question within some qualifying time frame;
? whether that person intends to buy within some time frame;
? whether that person is in the geographic market; and
? whether that person is an adult.

 Finally, population boundaries may be set by cost. For example, a telephone survey to measure opinions about a supermarket might be limited to certain area codes that are expected to account for most customers, even though customers who come from outside the area will be missed by this definition.

 Step 2: identify the sampling frame
After defining the target population, a frame of the population must be obtained before sampling can begin.

 A sampling frame is a list or system that identifies every member of the target population so that a sample can be drawn without the necessity of physically contacting every member of the population.

 It can be a list of names and telephone numbers, as in telephone surveys, an area map of housing or a list of addresses purchased from a mailing list supplier.

 It could also be a database. The frame defines the sampling unit, the unit used in the design of the sample.

 The frame, and therefore the sampling unit, may take the form of households, students, retail stores of a particular defined type (nature and size, for instance), businesses or transactions.

 However, lists are not always available. In such a situation, some sort of counting system must be used to keep track of population members and identify the selections; for example every fourth shopper could be selected.

 Step 3: choose the sampling method
There are two main types of sampling methods: probability sampling and non-probability sampling.

 Probability samples comprise samples in which the elements being included have a known chance of being selected.

 A probability sample enables sampling error to be estimated. This, in simple terms, is the difference between the sample value and the true value of the population being surveyed.

 A sampling error can be stated in mathematical terms: usually plus or minus a certain percentage. A larger sample usually implies a smaller sampling error.

 Non-probability samples are ones in which participants are selected in a purposeful way. The selection may require certain percentages of the sample to be women or men, housewives under thirty or a similar criterion.

 This type of selection is an effort to reach a cross-section of the elements being sampled. However, because the sample is not rigorously chosen it is statistically impossible to state a true sampling error.

 Types of Probability Sampling
Most samples chosen for applied research are non-probability samples.

A true probability sample, because of the stringent requirements, is likely to be far too expensive and too time-consuming for most uses.

 The sampling method chosen for any particular study, therefore, must be explained carefully, with the reasons for its acceptability and likelihood of supplying accurate data.

 The research plan may not require that a whole country be sampled. Cost and time factors may lead to the decision to cover only part of a country.

 Probability sampling
Simple random sampling is a technique in which each element of the population has an equal chance of being selected.

 Simple random sampling is carried out by assigning each element of the sampling frame a number.

 Then a series of random numbers is generated, using either a computer or random number table.

 The sample becomes the elements whose numbers appear on the list of random numbers. This method is appealing because it produces an unbiased estimate of the population’s characteristics.

 It guarantees that every member of the population has a known and equal chance of being selected; therefore, the resulting sample, no matter what the size, will be a valid representation of the population.

 To obtain a simple random sample is not easy or practical in many circumstances. It may be time-consuming or costly and sometimes is theoretically impossible.

 For instance, if we wish to take a simple random sample from a large finite population of one million families, although it is possible, it is not a simple task to assign a number to each of the families and then draw a sample at random from the numbers.

When a population is infinite, numbering each element is impossible. Therefore, simple random sampling needs modifications.

 The most common types of modified probability samples are systematic, stratified, and cluster samples.

 Systematic sampling is a technique in which a sample is drawn by choosing a beginning point in a list and then sequentially selecting every kth element from the list.

 For a systematic sample, the items in the population must be ordered. The selection procedure depends on the number of items in the population and the size of the sample.

 The number of items in the population is first divided by the number desired in the sample.

 The quotient is k, indicating whether every tenth, eleventh, or perhaps hundredth element in the population is to be selected.

The first item of the sample is selected at random.

The rest of the sample is chosen by selecting every kth element from the ordered list until the sample size is reached.

 The popularity of systematic sampling has fallen because computerized databases now have a random number selection capability.

 However, in the special case of a physical listing of the population, such as a membership directory or a telephone book, systematic sampling is often chosen over random sampling because of its economic efficiency.

 In this instance, systematic sampling can be applied with less difficulty and accomplished in a shorter time period than can simple random sampling.

 Furthermore, systematic sampling has the potential to create a sample that is almost identical in quality to samples created from simple random sampling.

 The essential difference between systematic sampling and simple random sampling is apparent in the use of the words “systematic” and “random.”

 The system used in systematic sampling is the skip interval, whereas the randomness in simple random sampling is determined through the use of successive random draws.

 In cluster sampling, a random sample of subgroups is chosen and all members of the subgroups become part of the sample.

It is done by first dividing the population into subgroups.

Then a random sample of subgroups is chosen, and all members of the chosen subgroups are included in the study.

 Notice here that not all subgroups are selected, but those that are selected compose the sample.

 If the researcher samples all of the members of the selected subgroups, it is a one-stage cluster sample.

 If a sample of members of the selected subgroups is randomly selected, it is a two-stage cluster sample.

 One of the most popular ways of forming a cluster is by geographic areas. For the first step, the researcher could select a random sample of areas, and then for the second step pick a probability method to sample individuals within the chosen areas.

 The two-step area sample approach is preferable to the one-step approach because there is always the possibility that a single cluster may be less representative than the researcher believes. But the two-step method is more costly because more areas and time are involved.

 The greatest danger in one-stage cluster sampling is cluster specification error that occurs when the clusters are not homogeneous.

 In stratified sampling, the researcher first divides the population into natural sub-groups that are more homogeneous than the population as a whole.

 Then items are selected for the sample at random or by a systematic method from each subgroup.

 This method is usually used when a large variation exists within a population and the researcher has some prior knowledge about natural subgroups within the population.

 Estimates of the population based on the stratified sample usually have greater precision (or smaller sampling error) than if the whole population were sampled by simple random sampling.

The number of items selected from each stratum may be proportionate or disproportionate to the size of the stratum in relation to the population.

 Under the proportionate method, for example, if the size of stratum A is 30 percent of the population, then 30 percent of the sample will come from stratum A. So, if the sample has 300 items.

 30 percent of the sample size, or 90 items, are to be selected from stratum A. When the selection is disproportionate, it is relatively difficult to weigh the results from individual strata properly.

 The main benefit of stratified sampling is that the sample will include items from each stratum.

 The above table shows that if the stratified sample of companies is proportionate, the large companies are only represented with one company in the sample of 111 companies.

 Taking the importance of the large companies into consideration, the proportion of large companies should be higher.

 Therefore, in this case represented in the above table we would prefer a disproportionate sample.

 There are times when stratified sampling is used in marketing research because skewed populations are encountered.

Prior knowledge of populations under study, augmented by research objectives sensitive to subgroupings, sometimes reveals that the population is not normally distributed.

 Under these circumstances, it is advantageous to apply stratified sampling to preserve the diversity of the subgroups.

Usually, a surrogate measure, which is some observable or easily determined characteristic of each population member, is used to help partition or separate the population members into their various subgroups.

 Non-probability sampling methods
All of the sampling methods described so far embody probability sampling assumptions. In each case, the probability of any unit being selected from the population into the sample is known, even though it cannot be calculated precisely.

 Convenience samples are samples drawn at the convenience of the interviewer. The selection of place and, consequently, prospective respondents, is subjective rather than objective.

Certain members of the population are automatically eliminated from the sampling process.

 When researchers have little time or money available for an elaborate study, they may do convenience sampling, selecting sample items that are easy to obtain.

 In fact, there may be no other way to gather data in some cases than to sample a group of individuals who are available.

Often, for example, college professors will use their students as a sample, because students are a captive audience and are convenient for the study.

 The problem here lies with the subjective selection of the sample and the lack of generalizability of the results (Wansink, 2002).

 Judgement samples differ from convenience samples because they require a judgement or an “educated guess” as to who should represent the population.

 Often the researcher or some individual helping the researcher who has considerable knowledge about the population will choose those individuals whom they feel constitute the sample.

It should be apparent that such samples are highly subjective and, therefore, prone to error. Focus group studies often use judgement sampling rather than probability sampling.

Quota samples establish a special quota for various types of individuals to be interviewed.

 Researchers may want to ensure that their sample includes a sufficient number of individuals with a particular characteristic that affects the study.

 In these cases the researchers determine the percentage of the target population that possesses the characteristics of interest and specify the number of these individuals to be included in the sample to reflect their proportion in the population.

 Quota samples are often used by companies that have a firm grasp on the features characterizing the individuals they wish to study.

 A large bank, for example, might stipulate that the final sample have an equal number of males and females because in the bank’s understanding of its market, the customer base is equally divided between the two sexes.

 When done conscientiously and with a firm understanding of quota characteristics, such sampling can rival probability sampling in the minds of researchers.

 Snowball samples require respondents to provide the names of additional respondents. Such lists begin when the researcher compiles a short list of sample units that is smaller than the total sample for study.

 After each respondent is interviewed, they are asked to name other possible respondents. In this manner, additional respondents are referred by previous respondents.

 Or, as the name implies, the sample grows just as a snowball grows when it is rolled downhill.

 Snowball samples are most appropriate when there is a limited and disappointingly short sample frame and when respondents can provide the names of others who would qualify for the survey.

 The non-probability aspects of snowball sampling come from the selectivity used throughout.

 The initial list may also be special in some way, and the primary means of adding people to the sample is by tapping the memories of those on the original list. Referral samples are often useful in industrial marketing research situations.

 Step 4: determine the sample size
Having chosen the sampling method, the next step is to determine the appropriate sample size.

 If the sample size is too large, more money and time will be spent than is necessary, but the result obtained from the large sample may not be more accurate than that from a smaller sample.

 On the other hand, if the sample size is too small, the study may not reach a valid conclusion.

 It is important to realize that the more elements that are properly sampled from the population, the less the sampling error.

 This error exists because the whole population is not examined, ultimately leaving something out of the investigation.

 The most correct method of determining sample size is the confidence interval approach, which applies the concepts of accuracy (sample error), variability, and confidence interval to create a “correct” sample size.

 Because it is, theoretically, the most correct method, it is the one used by national opinion polling companies.

To describe the confidence interval approach to sample size determination, we first describe an underlying concept.

 The larger a probability sample is, the more accurate it is (less sample error), indicating that there is a relationship between sample size and the accuracy of that sample.

 However, the relationship between sample size and accuracy is not linear – doubling the sample size

95% confidence intervals (= sample error = E) obtained around estimates of proportions, given various sample sizes does not halve the sample error.

I

n fact, the sampling error diminishes in accordance with the square root of the growth in sample size.

So if the sample doubles, the sampling error decreases by a little more than 40 percent. Because of this statistical relationship you have to quadruple a sample in order to halve the sample error. The above table dipicts this principle.

Step 5: gather the data
Gathering data is a two-stage process. First, the sample unit must be selected. Second, information must be gained from that unit.

 Simply put, you need to choose a person and ask him or her some questions. However, not everyone will agree to answer.

So there comes the question of substitutions. Substitutions occur whenever an individual who was qualified to be in the sample proves to be unavailable, unwilling to respond, or unsuitable.

 The final activity in the sampling process is the assessment stage. Sample assessment can take a number of forms, one of which is to compare the sample’s demographic profile with a known profile such as the census.

 With quota sample validation, the researcher must use a demographic characteristic other than those used to set up the quota system.

 The essence of sample validation is to assure the client that the sample is, in fact, a representative sample of the population about which someone wishes to make decisions.

 summing up
Sampling design begins by defining the target population in terms of elements, sampling units, extent and time.

Then the sampling frame should be determined. A sampling frame is a representation of the elements of the target population.

 At this stage, it is important to recognize any sampling frame errors. The next step involves selecting a sampling technique and determining the sample size.

 In addition to quantitative analysis, several qualitative considerations should be taken into account in determining the sample size.

 Execution of the sampling process requires detailed specifications for each step. Finally, the selected sample should be validated by comparing characteristics of the sample with known characteristics of the target population.

 Sampling techniques may be classified as either non-probability or probability techniques.

 When conducting international marketing research it is desirable to achieve comparability in sample composition and representativeness even though this may require the use of different sampling techniques in different countries.

 Non-probability sampling is based on researchers’ subjective judgement rather than on scientific principles.

 However, this does not mean the results are useless. On the contrary, a researcher may do a good job in portraying the target population, but without scientifically determined samples, there is no way to determine how precise the results are.

 But the ease of obtaining the sample and the low cost associated with drawing non-probability samples often compensate for their lack of statistical support.

 Judgement sampling, convenience sampling, quota sampling, and snowball sampling are popular non-probability sampling methods.

 Probability sampling is any sampling plan in which the chance of being selected is known and equal for every sampling unit in the population.

 Statisticians prefer these methods since sampling selection is objective and the sampling error may be measured.

Simple random sampling, systematic sampling, stratified sampling, and cluster sampling are types of probability sampling methods.

 For a sample to be statistically useful, it must be representative of the target population. While industry rules of thumb, affordability, and statistical methods can all be used to determine sample size, the statistical method is preferred, because it is supported by scientific principles.

 Using this method, researchers need three pieces of information: desired precision, desired confidence level, and an estimation of the population standard deviation or parameter.

 Describe the sampling design process.

Distinguish between probability and non-probability samples. What are the advantages and disadvantages of each?

Why are non-probability samples popular in marketing research?

 Describe snowball sampling. Give an example of a situation in which you might use this type of sample. What are the dangers associated with this type of sample?

 What are the differences between proportionate and disproportionate stratified sampling?

 What is the least expensive and least time-consuming of all sampling techniques?

 What is meant by a “skewed” population? Illustrate what you think is a skewed population distribution variable and what it looks like.

 Differentiate one-step from two-step area sampling, and indicate when each one is preferred.

 Discuss the factors that determine sample size.

 Keywords: sampling, probability sampling, non-probability sampling, target population, sampling frame, probability sampling, non-probability sampling, sampling methods, sampling error, Simple random sampling, Systematic sampling, cluster sampling, Convenience samples, Judgement samples, Quota samples, Snowball samples, sample size,

Report This Post

The questionnaire in quantitative research is used as a survey instrument with larger samples, normally containing structured questions for ease of coding and analysis.

A questionnaire is a data collection instrument, formally setting out the way in which research questions should be asked.

Even simple questions need proper wording and organization to produce accurate information.

Consideration needs to be given to how questions should be worded, in the light of the objectives of the research, and the target group of respondents. Attention also needs to be given to the organization of the questionnaire and to testing.

There are limitations to what a questionnaire can measure, especially when it comes to product recalls because people can only accurately report upon the item they have bought for a limited time after the actual purchase.

A questionnaire has specific objectives:
? To translate the information needed into questions that the respondents can and will answer. Developing questions that will yield the desired information is difficult.

Two apparently similar ways of posing a question may result in different information. Hence, this objective is a challenge.

? A questionnaire must motivate the respondent to become involved in the interview, to co-operate, and to complete the interview.

Before designing a questionnaire, the researcher must evaluate “what is the respondent going to get out of this” and appreciate what respondents go through when approached and questioned.

Not all respondents are the same in what they seek from a questionnaire or interview process.

? A questionnaire should minimize response error. Such errors arise when respondents give inaccurate answers or when their answers are mis-recorded or mis-analyzed.

A questionnaire can be a significant source of response error and minimizing this error is an important objective of questionnaire design.

To develop a further understanding of questionnaire design, the process can be presented as a series of steps.

The process outlined below shows that the ten steps are interrelated and the development of a questionnaire involves much iteration and interconnection between stages.

Step 1: specify the information needed
The research process often begins when the international marketing manager, brand manager, or product development specialist has a need for decision-making information that is not available.

Five classes of information, generally useful for marketing decisions have been identified:

? Facts and knowledge: what are the beliefs, perceptions, and depth of knowledge of the survey respondents about, for example, specific products, services, industries, or organizations?

? Opinions: what are the existing attitudes towards products, etc., including an assessment of the strength with which these attitudes are held?

? Motives: what motivates buyers of various kinds of products or services?

? Past behaviour: what are the patterns of consumption over specified time periods? Insight will be given into factors such as brand loyalty.

Observational techniques, such as home audits, will help in verifying reported consumption behaviour.

? Future behaviour: indications of possible behaviour may be gleaned from sensitive questioning.

Included in this approach, of course, would be an evaluation of the nature of buying behaviour, which may be complex, and making a list of research objectives.

The first step in questionnaire design is to specify the information needed. It is helpful to review the components of the problem and the approach, particularly the research question, hypotheses and characteristics that influence the research design.

To further ensure that the information obtained fully addresses all the components of the problem, the researcher should prepare a set of variables, which would influence the decision-making problem.

The project’s research objectives should then be translated into information goals that are specific enough to guide question formulation, thinking ahead to the types of tables and graphs that can be used in the final report.

Clearly define the target respondents. The characteristics of the respondent group have a great influence on questionnaire design.

The wording and style of questions that may be appropriate for finance directors being surveyed about their IT needs may not be appropriate for retired persons being asked about their holidays.

The more diversified the respondent group, the more difficult it is to design a single questionnaire appropriate for all.

Step 2: determine the survey methods
A telephone interview often requires a rich verbal description of a concept to make certain the respondent understands the idea being discussed.

In contrast, in an interview, the researcher can show the respondent a picture or demonstrate the concept.

If you want respondents to react to physical stimuli, face-to-face interviews are best. Telephone surveys are very limited in this regard, though it may be possible to send materials to respondents in advance of a phone interview.

Step 3: determine question type and content
Two main types of question response formats are used in marketing research. Openended questions are those in which respondents can reply in their own words.

In other words, the researcher does not limit the response choices. The advantage of such responses is that they can provide the researcher with a rich array of information.

The respondent is answering from his own frame of reference. Advantages are described in “real world” terminology rather than marketing jargon.

Often this is helpful in designing promotional themes and campaigns. It enables copywriters to use the consumer’s language. This rich array of information can now be captured even in computer-assisted interviews.

Open-ended questions are not without their problems. Editing and interpretation of responses are problematic.

If too many categories are used, data patterns and response frequencies may be difficult for the researcher to interpret.

If the categories are too broad, the data are too general and meaning may be lost. Even if a proper number of categories is used, editors may have to interpret what the interviewer has recorded and force data into a category.

A related problem of open-ended questions is interviewer bias. Although training stresses the importance of verbatim recording of open-ended questions, it is often not practised in the field. Also, slow writers may miss important comments.

Open-ended questions also may be biased towards the articulate interviewee. A person with elaborate opinions and the ability to express them may have much greater input than a shy, inarticulate, or withdrawn respondent. Yet they could be equally likely prospects for a product.

So, a basic problem with open-ended questions lies in interpreting and processing data. In fact, a two-phase judgement must be made.

First, the researcher must decide on the proper set of categories and then each response must be evaluated as to which category it falls into.

In general, open-ended questions are useful in exploratory research and as opening or closing questions.

They should be chosen with care as their disadvantages can outweigh their advantages in a large survey.

Closed questions require the respondent to make a selection from a list of responses. The primary advantage of a closed question is simply the avoidance of many of the problems of open-ended questions.

Interviewer and coder bias are removed because the interviewer is simply checking a box, circling a category, recording a number, or punching a key.

The simplest form of a closed-ended question is the dichotomous choice. An example is:
Do you think that this year’s inflation will be greater or less than last year?

? Greater than last year
? Less than last year

Note that the respondent is limited to two answers. It is easy to administer and usually evokes rapid response.

Often a neutral or no opinion/don’t know is added to dichotomous questions to take care of those situations.

Dichotomous questions are prone to measurement error. Because alternatives are polarized, possible choices between the poles are omitted. Thus, question wording is critical to obtain accurate responses.

In the above question, response may vary depending upon whether greater than or less than is listed first.

These problems can be overcome using a split ballot technique. Onehalf of the questionnaires are worded with greater than listed first and the other half with less than first. This procedure will aid in the reduction of potential bias.

As well as open-ended and closed questions, there are multiple-choice questions. These overcome many of the disadvantages of open-ended questions because interviewer bias is reduced and the questions are administered quickly.

Also, coding and processing of data is much less costly and time-consuming. In self-administered questionnaires, respondent co-operation is improved if the questions are structured.

In multiple-choice questions, the researcher provides a choice of answers and respondents are asked to select one or more of the options given.

Sometimes the respondent is asked to only select one item. For instance, a question might ask the respondent “What is your preferred brand?

Alternatively, it could ask the respondent to indicate, say, three favourite brands. A third way would be to simply let the person make the choices that are regarded as relevant or that apply. Consider the following example:

Please indicate all the brands of soft drinks that you have consumed in the past week.
Please check all that apply.

1. Coca-Cola       –
2. Pepsi-Cola      –
3. Fanta              –
4. Seven Up       –
-
-
-
14. Dr pepper       –
15. Other
(please spsescify  –

Of concern in designing multiple-choice questions is the number of options and the order of potential responses, known as position bias.

The response options should include all possible choices. The general guideline is to list the important options and to include another labelled “Other (please specify),” as shown above. The responses should be mutually exclusive.

Multiple-choice questions are not without disadvantages. Considerable effort is required to design effective multiple-choice questions and qualitative techniques may be required to determine the appropriate response options.

It is difficult to obtain information on items not listed. Even if an “Other (please specify)” category is included, respondents tend to choose from the list.

In addition, showing respondents the list of answers produces biased responses. There is also the potential for order bias.

The choice between open- and closed-response questions is not necessarily an either/or distinction.

Open-response questions can be used with closed-response questions to seek additional information.

Using an open-response question to follow up a closed question is called a probe. Probes can combine some advantages of both open and closed questions.

They can be used for specific pre-chosen questions or to obtain additional information from only a subset of people who respond to previous questions in a certain way.

A common example of the latter is to ask respondents who choose “none of the above” a follow-up question to expand on their answer.

There are two general purposes for the use of probes in a questionnaire. The first is to pinpoint questions that were diffiult for respondents.

Adequate testing of questions reduces the need for probes. The second purpose is to aid researcher interpretation of respondent answers.

Answers to open-response follow-ups can provide valuable guidance in the analysis of closedresponse questions.

Step 4: establish question format
Every question in a questionnaire should contribute to the information needed or serve some specific purpose.

It is useful to ask some neutral questions at the start to establish involvement and rapport, particularly when the topic of the questionnaire is sensitive or controversial.

Sometimes filter questions are asked to disguise the purpose or sponsorship of the project.

For example, rather than limiting the questions to the brand of interest, questions about competing brands may be included. At times, certain questions may be duplicated to assess reliability or validity.

Once a question is deemed necessary, the researcher must make sure that it is sufficient to get the desired information.

Sometimes several questions are needed to obtain the required information in an unambiguous manner.

A double-barrelled question is really two different questions posed in one question. With two questions posed together, it is difficult for a respondent to answer either directly.

Consider a question asked of patrons at a restaurant “Were you satisfied with the food and service?” How does the respondent answer?

If they say “yes” does that mean they were satisfied with the food? The service? A combination?

The question would be much improved by asking about a single item: one question for food and another question for service.

Step 5: choose question wording
The wording of specific questions always take a significant amount of time. It is a skill developed over time and subject to constant improvement.

Questions are the raw material of questionnaires and vital to the quality of the research. Question wording is the translation of the desired question content and structure into words that respondents can clearly and easily understand.

Deciding on the wording is perhaps the most critical task in developing a questionnaire. If a question is worded poorly, respondents may refuse to answer it or answer incorrectly.

Even small changes in wording can shift respondent answers, but it is difficult to know whether or not a wording change will have such an effect.

Question phrasing depends on such factors as the information being sought, the characteristics of target respondents, and where the survey is administered. Good questionnaire writing requires that researchers follow these guidelines:

? The questions should be easy to understand. Ordinary words should be used in a questionnaire, and they should match the vocabulary and intellectual level of the respondents.

The most common pitfall is to use technical jargon or specialized terms. Special care must be taken to avoid words that have different meanings for different groups.

This can be readily appreciated in cross-cultural studies, where translation problems are profound.

One socio-economic group may refer to the evening meal as dinner, while others call this meal supper and have their dinner at noon.

Most respondents do not understand technical marketing words. Never forget that you are imposing your language upon respondents in the form of a questionnaire.

Your language communicates and puts respondents in a particular frame of mind as they answer the questions you pose.

Unless that language is meaningful to respondents, they will be put in a frame of mind that you may not intend, and be answering different questions from those you set.

? Questions should be focused on a single issue or topic. The researcher must stay focused on the specific issue or topic.

The question “What type of hotel do you usually stay in when on a trip?” is too vague.

A more focused version is “When you are on a family holiday and stay in a hotel at your destination, what type of hotel do you typically choose?”

? The question should be a grammatically simple sentence. A simple sentence is preferred over compound and complex sentences.

The more complex the sentence, the greater the potential for respondent error. To avoid these problems, the researcher should strive to use a simple sentence structure, even if two separate sentences are necessary for the question.

What is an appropriate length of a question? A common rule of thumb is to limit the number of words in any question to under 20.

Under certain circumstances, a question may have to be long in order to avoid ambiguity, but this should be the exception.

A questionnaire filled with long questions is tiring to answer and more difficult to understand.

Brevity will help respondents to comprehend the central question and reduce the distraction of wordiness.

? Avoid leading questions. A leading question is one that suggests the answer or reveals the researcher’s (or interviewer’s) opinion.

This can be done easily by adding “don’t you agree?” or “wouldn’t you say?” to a desired statement.

A loaded question introduces a more subtle bias. A common type of loading of possible responses is through failure to provide a full range of options, for example, by asking, “How do you generally spend your free time-watching television, or what?” Simply adding “I’m sure you agree” or “Don’t you think?” to a statement can bias responses.

Researchers have also found that respondents tend to agree with plausible propositions unless they have a strong opinion or choices are provided.

Even when options are offered, respondents tend to agree rather than disagree with plausible statements. Given this tendency, questions should be worded neutrally.

? Consider the ability of the respondent to answer the question. Asking respondents about a brand or store that they have never encountered creates a problem.

When a question is worded in such a manner that it implies that the respondent should be able to answer it, then often a reply will be forthcoming, but it will be nothing more than a guess.

This creates measurement error, since uninformed opinions are being recorded. A second problem is forgetfulness.

For instance, “What was the name of the last film you saw on TV?” “Who were the stars?” To avoid the problem of a respondent’s inability to remember, time periods should be kept relatively short.

? Consider the willingness of the respondent to answer the question. Reporting of an event is likely to be distorted in a socially desirable direction.

If the event is perceived as embarrassing, sensitive in nature, threatening, or divergent from one’s self-image, it is likely either not to be reported or to be distorted.

Embarrassing topics that deal with things such as borrowing money, personal hygiene, sexual activities, and criminal records must be phrased in a careful manner to minimize measurement error (Wrobel, 2002).

One technique is to ask the question in the third person. For example, “Do you think that most people spend more using their credit cards than they should?”

By asking about “most people” rather than about the respondents themselves, researchers may be able to learn more about the respondents’ attitude to credit and debt.

A third method for soliciting embarrassing information is to state that the behaviour or attitude is not unusual before asking the question.

For instance, “Millions of people suffer from hemorrhoids; do you or any member of your family suffer from this problem?”

This technique is called “using counter-biasing statements,” and makes embarrassing topics less intimidating for respondents to discuss.

Step 6: arrange the sequence and layout of questions
After questions have been formulated, the next step is to order them and develop a layout for the questionnaire.

Questions should be asked in a logical order. All questions that deal with a particular topic should be asked before beginning another topic.

When switching topics, brief transitional phrases should be used to help respondents switch their train of thought.

“Branching” or “skipping,” a procedure in which certain questions are not asked if they do not apply to a respondent, should be designed carefully.

Branching questions direct respondents to different places in the questionnaire based on how they respond to the question at hand.

These questions ensure that all possible contingencies are covered. A simple way to account for all contingencies is to prepare a flowchart of the logical possibilities and then develop branching questions and instructions based on it.

Branching is most easily done in computer-assisted telephone interviews or online surveys, where software can perform the branching. The logical order of a questionnaire could be:

? Use screener questions to identify qualified respondents. Most market research employs some variation of quota sampling.

Only qualified respondents are interviewed, and specific minimum numbers (quotas) of various types of qualified respondents may be desired.

A study on food products generally has quotas of users of specific brands, a magazine study screens for readers, a cosmetic study screens for brand awareness and so forth.

Thus, any demographics obtained provide a basis for comparison against persons who qualify for the study.

A long screener can significantly increase the cost of the study. It means that you are obtaining more information from every contact with a respondent.

Short screeners quickly eliminate unqualified persons and enable the interiewer to move to the next potential respondent.

Yet a long screener can provide important information on the nature of non-users, or persons unaware of the product or service being researched.

? After introductory comments and screens to find a qualified respondent, the initial questions should be simple, interesting, and non-threatening.

Income or age questions might be disastrous. These are often considered threatening. The initial question should be easy to answer without much forethought.

? Ask general questions first. Once the interview proceeds beyond the opening “warmup” questions, the questionnaire should proceed in a logical fashion.

General questions are covered first to get the person thinking about a concept, company, or type of product, and then the questionnaire moves to the specifics.

? Ask questions that require effort in the middle of the questionnaire. Initially, the respondent is only vaguely interested and understanding of the nature of the survey, until the process builds momentum and commitment to the interview.

When the interviewer shifts to questions with scaled-response formats, the respondent must be motivated to understand the response categories and options.

Build interest and commitment early to motivate the respondent to finish the questionnaire.

? Put sensitive, threatening, and demographic questions at the end. Sensitive topics include money, personal hygiene, family life, political and religious beliefs, and involvement in accidents or crimes.

In industrial surveys, sensitive questions may encompass much of what a company does, especially if it reveals strategy and plans.

Placing these questions at the end ensures that most of the questions will be answered before respondents become defensive or break off the interview.

Moreover, rapport has been established between the respondent and the interviewer by this time, increasing the likelihood of an answer.

Conclude the survey by thanking respondents for their time. You may also want to inform respondents how they can obtain the results of the survey.

The format, spacing and positioning of questions can have a significant effect on the results, particularly in self-administered questionnaires.

It is good practice to divide the form into parts. Several parts may be needed for questions pertaining to the basic information.

Each questionnaire should be numbered. This helps the control questionnaires in the field as well in coding and analysis.

Numbering makes it easy to account for the questionnaires and to determine whether any have been lost.

Finally, don’t make the questionnaire too long. Long questionnaires are tiring and overwhelm respondents.

As respondents exceed the time they have mentally set aside to complete the survey, their responses are no longer accurate. Furthermore, long questionnaires tend to have high non-response rates.

Step 7: obtain approval from relevant parties
Copies of the draft questionnaire should be distributed to all parties that have direct authority over the project.

The client is given the opportunity to comment during the client approval stage, in which the client reviews the questionnaire and assesses whether it covers all of the appropriate issues.

If questions are either inappropriate or perhaps can be improved, it is necessary for the client to convey these changes to the researcher.

This may cause changes, but it is important for the client to approve the questionnaire that will be used.

Client approval ensures that the client is aware of the survey’s progress, and the initialled questionnaire ensures that the researcher is protected against any later claims that the questions were incomplete or done incorrectly.

Step 8: test, revise and correct problems
Once approval has been obtained, the questionnaire should be tested to identify and eliminate problems.

The most basic test is to have as many people as possible look at drafts of the questionnaire as a sounding board. The worst problems will be uncovered by these reviews.

Ideally, a pilot test is done by the best interviewers who will ultimately be working on the job and is administered to target respondents for the study.

They are told to look for misinterpretations by respondents, lack of continuity, poor skip patterns, additional choices for pre-coded and closed questions, and general respondent reaction.

Testing could also involve a trial run of the questionnaire using a small sample of respondents, say five to ten, from the target population.

While the sample may be small, it should cover all subgroups of the target respondents. The goal of testing is to check that the questionnaire will capture the information sought by the researcher.

Testing helps refine the instrument and identifies errors that may be apparent only to the target.

Testing is usually done in two stages. The first stage is personal interviews, regardless of the way the questionnaire will later be administered, because researchers need to observe the behaviours of both the interviewers and the respondents.

Respondents’ reactions to the questions are the primary interest. The interviews can be carried out through protocol analysis or debriefing.

Protocol analysis is an interviewing technique in which respondents think aloud while responding to each question.

Debriefing is an interview conducted after respondents have completed the questionnaire. The respondents are then informed that the exercise was a test and are asked to share with the researcher their thoughts about the question, their answers, and any shortcomings of the survey.

The second testing stage involves administering the survey to a small sample in an environment as similar as possible to the one in which the questionnaire will ultimately be administered.

This stage often reveals problems that cannot be detected in personal interviews. In either phase, researchers should attempt to eliminate any problems and revise the questionnaire.

After the questions have been checked, a numerical code is allocated to each type of response to aid data processing.

All possible answers may be listed and coded in advance of the interview and, in surveys of any size, this is done wherever possible.

If responses cannot be allocated to a range of possible answers, coding can take place after the interview. Especially, pre-coding is relevant in quantitative surveys with closed questions.

Step 9: prepare final copy
Even the final copy phase does not allow the researcher to relax. Precise typing instructions, spacing, numbering, and pre-coding must be set up, monitored, and proof-read.

In general, the quality of copying and the paper used is a function of who will see the questionnaire.

Step 10: implementation
Most research interviewing is conducted by a field services department. It is their duty to complete the interviews and send them back to the researcher. In essence, field services are the production line of marketing research.

Supervisor’s instructions inform interviewers of the nature of the study, start and completion dates, quotas, reporting times, equipment and facility requirement, sampling instructions, number of interviewers required, and validation procedures. Detailed instructions are required for any taste test that involves food preparation.

The supervisor’s instructions are vitally important. Without clear instructions, the interview may be conducted in ten different ways in ten different countries.

Call record sheets are used to measure the efficiency of the interviewers. A form normally indicates the number of contacts and the results of the contact.

A supervisor can examine calls per hour, contacts per completed interview, average time per interview, and similar measures to analyze an interviewer’s efficiency.

If, for example, contacts per completed interview are high, the field supervisor should examine the reasons behind it. Perhaps the interviewer is not using a proper approach or the area may be difficult to cover.
What is the purpose of a questionnaire?

How would you determine whether a specific question should be included in a questionnaire?

What are the issues involved in designing multiple-choice questions?

What are the guidelines available for deciding on question wording?

What is a leading question? Give an example.

Once a questionnaire is developed, what other factors need to be considered before giving it to interviewers?

Why is testing a questionnaire important?

Keywords: Questionnaire, semi-structured Questionnaire, unstructured Questionnaire, Openended questions, Closed questions, split ballot technique,

Report This Post