Chapter 6 - surveys and observations

Surveys and polls

• Mean the same thing

• A method of asking people questions in a standardized format 

  • face to face, written questionnaires, online

open-ended questions

people are asked a question that allows them to answer in any way they like 

Forced choice questions

people provide their opinion by selecting the best of two or more options

Likert scale

people presented with a statement and must indicate their degree of agreements with the statement

• Ex. on a scale of 1-5

  • 1 (strongly disagree) - 5 (strongly agree)

• If survey/question is similar to likert scale but differs in one or more dimensions (1 type of time, different anchors/opinions) → called likert-type scale

  • Ex. 1 (easy) - 5 (hard)

  • Questions responses can be set up by question structure

• Ex. if question premise is based on “agree”

• Ex. if questions premise is based on “disagree”

Semantic differential format

people are asked to rate a target using a numeric scale anchored by adjectives

• Ex. difficulty: 1 (show up and pass) - 5 (hardest thing i've ever done)

Leading questions

questions whose wording can be biased toward making a particular response

• Goal is to capture true opinions → questions should be worded as natural as possible 

Loftus and palmer (1974)

• Participants were asked how fast cars were going when they smashed each other in a video

• Leading questions and words can bias ppl to respond a certain way → can influence recollection instances

Double barreled question

• question that is actually asking two separate things

  • Ex. do you enjoy swimming and wearing sunscreen?

    • Poor construct validity bc researchers cant be sure which one you are asking to measure/responding to 

Negatively worded question

• questions with negative phrasing, can lead to double negatives

  • Ex. Do you believe that people who do not drive with a suspended license should never be punished?

  • Poor construct validity bc researchers cant be sure what the question is asking bc it's too confusing 

Question order and context 

• The order questions are asked in can influence how people response

• Earlier questions can influence how later questions are framed 

  • Ex. participants can be pried to think about subsequent questions in a different way 

• Miller and krosnick (1998)

  • Found that voters are 5% more likely to vote for the first person on the ballot 

  • How can you control this?

    • Multiple versions of a survey with different question orders that allow you to test for different order effects

Encouraging accurate responses

• Do we trust self report?

  • People can give accurate and meaningful responses

    • But need to keep biases in mind

• Self reporting “more than they can know” 

  • People dont lime answering “idk” so they can make a reasonable guess and pass it off as the truth

• Self reporting memories of events

  • Unreliable and can change

• Rating consumer products

  • Ratings can be biased by who bothers to rate, cost of item, and more

Response set (non-differentiation)

• people respond to a number of questions in the same way without considering the questions themselves 

• Ex. response “strongly agree” to all questions 

• Weakens construct validity → no accurate measure of the respondents opinion 

2 types of response sets

1. Acquiescence (yea-saying): people answer positively instead of looking at the question

   1. Solution → word the same question in opposite ways

2. Fence sitting: people answer neutrally by selecting the center option in a scale

   1. Solution → remove neutral options and use forced choice

Socially desirable responding/faking good

trying to look better than they are (smarter, stronger thinner, etc)

Faking bad

trying to look more aggressive, deviant, nastier than they are 

Observational research

• Behaviour is observed and systematically recorded

• Can be better than self report → be cautious - bad observers cna have bad observations 

• Can be used frequency association, and causal claims 

Making reliable and valid observations 

• Construct validity can be threatened by:

  • Observer bias: observers see what they expect to see

  • Observer effects: participants change their behaviour to match observer expectations 

    • Ex. expectancy effects

  • Reactivity: participants change their behaviour as a result of that behaviour being observed 

• Examples:

  • Observer bias:

    • Being told either positive or negative information on the interviewee can influence opinion on the person 

  • Observer effects:

    • Bright and dull rats

    • Clever hans 

  • Reactivity

    • Jane goodal 

Preventing observer bias and observer effects

• Masked (blind) design: study design in which the observers do not know which condition a participant has been assigned to 

  • Ideally, observers don't know what study is about

  • Participant should be in the dark as well

  • Ex. single and double blind studies 

Preventing reactivity 

• Wait until person being observed becomes accustomed to the observer

• Use unobtrusive measures

  • Use one-way observation window or participant observer

  • Person who normally in the setting where the target behaviour occurs 

  • Ex. boss pretends to be a new hire to see the usual behaviour

• Measure product of the behaviour 

• Ethics → depends on what the situation is to determine if it is ethically ok to observe people 

Interviews

• Count and qualitative methods

  • Researches prepare a series of on-topic questions 

  • Responses recorded/transcribed

  • Analysis for general trends and patterns

  • Informants: ppl who are experts in a topic of interest → can be interviewed to capture their opinions, views, etc 

Focus groups

• Interviews conducted on small groups (~6 - 10)

  • Participants have relevant experience to a topic of interest

  • Ex. parents of preschoolers for new children's show

• Selection of group/sampling is important

  • Do they represent the population of interest?