Survey Research ()Social Research Methods2217 & 6501Fall, 2006

Part I: The Overview of Survey Research Research questions appropriate for a survey a brief history of survey research the logic of survey research

Survey: the most widely used data-gathering technique in the social sciences and in related fields easy to conduct surveys? No! Good surveys require thought and effort []

Research Questions Appropriate for a Survey Self-reported belief or behaviorsBehavior ()Attitudes/beliefs/opinions (//)Characteristics ()Expectations ()Self-classification ()Knowledge ()Be careful about why questionsIf ask respondents subjective understanding, why questions are appropriate

A History of Survey ResearchCan be traced back to the census ()Several reasons to explain the popularity of survey research (post-1950 growth)ComputersOrganizations (at universities)Data storageFundingMethodology: substantial research efforts applied

The Logic of Survey ResearchThe survey: sample many respondents who answer the same questionsTest multiple hypotheses and infer temporal orderCorrelational (): use control variables to approximate the causality test

Steps in Conducting a Survey:Develop an instrument (a survey questionnaire or interview schedule)Conceptualization and operationalization write clear and complete questionsHow to record and organize data for analysisTo collect dataCoding and entering dataAnalyze the data & report the result

Conducting a survey: Survey research: complex and expensive requires coordination between many people and steps requires organization and accurate record keeping[]

Part II. Constructing the Questionnaire principles of good question writing aiding respondent recall getting honest answers open vs. closed questions wording issues questionnaire design issues

Principles of good question writingA good questionnaire forms an integrated whole.Have an introduction & nice flowTwo key principles:Avoid confusion ()Keep the respondents perspective in mind ()Question writing: an art & a scienceTakes skills, practice, patience, and creativity

Ten things to avoid when writing survey questions:Avoid jargon, slang, and abbreviations ()Avoid ambiguity, confusion, and vagueness ()Avoid the use of indefinite words or response categoriesAvoid emotional language and prestige bias ()Use neutral language

Ten things to avoid when writing survey questions:Avoid double-barreled questions ()A double-barreled question: two or more questions joins togetherAvoid leading questions ()Do not lead the respondent to choose one response over another by its wording

Ten things to avoid when writing survey questions:Avoid asking questions that are beyond respondents capabilities ()Phrase questions in the terms in which respondents thinkBe very clear in asking questions Avoid false premises ()Avoid asking about distant future intentions ()

Ten things to avoid when writing survey questions:Avoid double negatives ()Double negatives: confusingAvoid overlapping or unbalanced response categories ()Response categories should be mutually exclusive, exhaustive, and balanced

Aiding Respondent Recall ()Recalling events accurately takes time and effort. (Our memory is not that trustworthy!)The problems of telescopingNeed to customize questions and interpret results cautiously.Provide aids to respondents:Specific instructions Extra thinking timeProvide fixed time or location referencesSituational framingDecompositionLandmark anchoring

Getting Honest Answers: Sensitive Topics ()Respondents may be reluctant to answer questions on sensitive topics.

Getting Honest Answers: Sensitive Topics ()Respondents may be reluctant to answer questions on sensitive topics.Use different techniques to get honest answers:Establish a comfortable setting before askingUse enhanced phasing of questionsProvide contextual background informationAsk more serious actions firstSelf-administered questionnaire or web-based surveyRandomized response technique (RRT)

Getting Honest Answers: Social Desirability Bias ()Social desirability bias: respondents tend to overstate an attitude or behavior ()Why? Respondents want to make their reports conform to social norms.Phrase questions to give respondents face-saving alternatives.

Getting Honest Answers: Knowledge Questions ()Knowledge questions may be threatening.Many people have inaccurate factual knowledge.Researchers can improve by: Phrase questions carefullyUse a sleeper question ()Check for more details

Getting Honest Answers: Contingency Questions ()A contingency question: to avoid asking questions that are irrelevant for a respondent (: )A two- (or more) part question: the first questions are screen or skip questions

Open versus Closed Questions ( vs. ) Open-ended questions: unstructured, free response vs. Closed-ended questions: structured, fixed responseEach has advantages and disadvantages (see Box 10.5, p. 287)

Open versus Closed Questions ( vs. )

Open versus Closed Questions ( vs. )

Open versus Closed Questions ( vs. ) Open-ended questions: unstructured, free response vs. Closed-ended questions: structured, fixed responseEach has advantages and disadvantages (see Box 10.5, p. 287)Large-scale surveys used more closed-ended questions because they are easier and quicker to answer.We can mix open-ended and closed-ended questions, and use partially open questions. Be careful about responses for closed-ended questions.

Neutral Positions: should provide a no opinion choice?Reasons for and against a neutral or middle positionThree kinds of questions: standard-format, quasi-filter, and full-filter questions ()Floaters (): respondents float from giving a response to not knowingAlso consider recency effect ()Either offer a middle position or make respondents to choose a position and then ask about their choices.Selective refusals for sensitive issues

Agree/Disagree, Rankings or Ratings ()?Questions or statements ()? Rate or rank-order items?Questions about values often show little differentiation rank-then-rate procedureBetter to rank items than rating ()Be careful about visual presentation and answer categories